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I.  INTRODUCTION 

This  thesis  deals  with  detection  and  estimation  using  spatially  separated  sensors. 
A  typical  practical  situation  is  a  surveillance  system  [1]  in  which  a  large  number  of 
sensors  monitor  some  region  of  space,  earth  or  sea  and  report  their  findings  to  a  global 
processor.  The  sensors  themselves  may  use  thermal,  acoustic  or  infrared  effects  to  form 
their  observations.  The  global  processor  performs  some  processing  on  the  data  to  come 
with  a  decision  or  for  taking  actions.  Because  of  many  considerations  such  as 
bandwidth  communication  limitations,  time  delay  or  because  the  amount  of 
information  is  too  massive  to  be  processed  by  a  single  processor,  the  processing  is 
carried  out  on  many  levels.  As  an  example  consider  the  case  of  distributed  detection. 
Detection  is  performed  at  the  sensor  level  and  at  the  fusion  center. 

Due  to  the  loss  of  information  in  the  local  processing,  the  overall  performance 
degrades.  However  a  great  communication  bandwidth  reduction  results.  If  the 
communication  channels  can  support  more  information  flow,  then  it  is  wise  to  perform 
"softer"  processing  at  the  local  level,  to  send  more  information  to  the  fusion  center,  and 
to  use  the  information  available  there  effectively. 

The  purpose  of  this  chapter  is  to  define  the  Distributed  Signal  Processing  (DSP) 
problem  in  general  and  to  show  some  reasons  and  situations  in  which  it  replaces 
Centralized  Signal  Processing  (CSP)  techniques.  We  then  will  review  the  status  of  the 
research  on  Decentralized  Detection  (DD)  problem,  one  of  the  basic  problems  of  DSP. 
Finally  the  contributions  and  organization  of  this  thesis  are  described. 

A.       OVERVIEW 

Classical  (Centralized)  Signal  Processing  (CSP)  assumes  complete  availability  of 
all  information  (signals)  at  one  central  processor  for  processing  (decision  making, 
computing,  detection,  estimation,  etc.).  While  this  situation  is  realistic  in  some  cases, 
many  real  world  systems  are  too  large  for  the  classical  processing  to  be  practically 
applied.  Power  systems,  detection  networks,  large  manufacturing  systems  and  military 
organizations  are  among  those  systems  in  which  total  centralized  signal  processing  is 
hard  to  apply.  Some  of  the  reasons  and  considerations  for  the  limations  of  CSP  are 
[2,3]: 
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1.  In  large  systems,  each  processor  has  partial  information  of  some  credibility. 
While  total  information  is  distributed  in  the  whole  system,  total  centeralization 
of  the  information  at  one  processor  is  impractical,  inconvenient  or  expensive 
due  to  limitations  in  the  system's  communication  channels,  memory  or 
computation  and  information  capabilities. 

2.  In  some  cases,  processing  speed  is  a  bottleneck.  Increasing  local  processing  of 
the  data  at  each  processor  and  sending  processed  data  to  the  next  level  of 
processors  will  help  relieve  the  problem. 

3.  When  reliability  of  the  system  is  of  major  concern,  distributed  processing  may 
better  tolerate  various  kinds  of  equipment  failures.  Less  complex  centralized 
processing  is  more  easily  shifted  to  a  new  location. 

4.  In  cases  when  security  is  a  major  problem,  increasing  local  processing  will 
decrease  the  information  handled  between  the  processors,  so  limit  any  other 
system's  access  to  the  process. 

5.  As  the  cost  of  computation  has  decreased  dramatically  relative  to  the  cost  of 
communication,  it  is  advantageous  to  trade  off  increased  computation  for 
reduced  communication.  So  in  Distributed  Sensor  Networks  (DSN)  involving 
geographically  distributed  sensors  that  collect  data,  it  may  be  more  economical 
to  locally  process  the  data  and  send  condensed  summaries  to  other  processors. 

Distributed  Signal  Processing  (DSP),  in  contrast  to  CSP,  has  several  processors 

that  cooperate  together  to  best  achieve  a  global  task  according  to  some  criterion.  A 

basic    problem    in    DSP,    which    has    attracted    much    attention    recently,    is    the 

Decentralized  Detection  (DD)  problem  (hypothesis  testing).    The  DD  problem  will  be 

a  major  concern  in  this  thesis.  A  summary  of  its  status  is  given  in  the  following  section. 

B.       MOTIVATION 

There  has  been  an  increased  interest  in  the  DD  problem  since  Tenney  and 
Sandell  introduced  it  in  1981  [5].  They  extended  the  classical  Bayesian  formulation  of 
the  detection  problem  to  distributed  environments.  Because  their  work  was  the 
pioneering  one  in  DD  and  because  we  will  refer  to  it  often  in  this  thesis  ,  let  us 
consider  it  now  in  some  detail  together  with  the  Centralized  Detection  (CD)  problem. 
Also,  because  detection  is  dealt  with  throughout  a  large  portion  of  this  thesis,  we  will 
make  some  remarks  about  the  phenomena  to  be  detected  and  about  detection  criterion. 
The  Phenomena 

Consider  observing  a  phenomena  H  of  M  possible  states  in  order  to  determine 
which  of  them  is  true.  For  M  =  2,  the  state  Hq  is  called  the  null  hypothesis  and  H^ 
the  alternative  hypothesis.   Their  probabilities  of  occurrence 

P(Ho)=Pq,       P(H^)=P^  (1.1) 
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are  assumed  to  be  known. 
The  Sensor  Observations 

The  phenomena  H  is  observed  by  N  sensors  SpS2,—,S>^  .  The  sensor 
observations  are  7^72,...., yJ^4  •  The  sensor  observations  have  known  conditional 
distributions 

p(yi.y2'-'yN/"o)'    p(yi.y2'-'yN/"i  )•  ^^-^^ 

Detection  Criterion 

The  function  of  the  detection  process  is  to  make  a  decision,  U^,  about  which 
state  of  the  phenomena  is  true.  The  optimality  criterion  is  a  function 

J:U^xH-*^,  (1.3) 

that  assigns  to  the  event  of  deciding  u-  when  H:  is  true  a  real  number  C.,  i,j  =  0,l, 
called  the  detection  cost,  so 

J{U^  =Ui,H  =Hj)=  Cj.  (1.4) 

The  objective  of  the  decision  rule  will  be  to  minimize  the  expected  decision  cost 
minE{J(u,H)}.  (1.5) 

An  important  ratio  in  our  analysis  is  the  constant  given  by 

C=   -foi£m£noL.  (1.6) 

Van  Trees  [6]  showed  that  the  average  decision  cost  is  given  by, 

R  =Coo  Pq  +C,,  P,  +Po  (Cq^  -C^^  )  ?,-?!  (Cqi  -C^^  )P,  (1.7) 
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where  P^  and  P^  are  the  probability  of  false  alarm^  and  probability  of  detection^ 
respectively.  At  this  point  we  will  make  the  assumptions  that 

Cqi    >     Cji  ,  (L8) 

and 

Cio   >     Cqo,  (1.9) 

These  assumptions  implies  that  it  is  more  costly  to  err  than  to  make  a  correct  decision. 
Equation  (1.7)  can  then  be  written  in  the  form: 

R=  r£ofLVt£oiIi  ^.  Pq  (^10 -Coo  )     p    p"[  p   ^r     .c,,  ).  (1.10) 

Ignoring   positive   constants  that  will  not  affect  our  analysis,  the  average  decision  cost 
R  is  given  by 

R  =  1  +  C  Pj.-Pj  .  (1.11) 

1.  The  Centralized  Detection  (CD)  Problem 

The  problem  of  centralized  binarv'  hypothesis  testing  can  be  posed  in  its  most 
general  form  as  follows.  For  the  structure  of  Figure  1.1  it  is  assumed  that  all  sensor 
observations  can  be  sent  to  one  (central)  location  for  processing.  The  function  of  the 
processor  is  to  map  the  vector  Y=  [y^  y^  ...  y^j  f  into  the  decision  space  U&subo(0,l) 

UJI-^(0,1)  (1.12) 


as  follows; 


0,    Hg  is  declared  to  have  been  detected 


^         r        0,  Hg 


U    =  ^  ^  (1-13) 

I     1,    H,    is  declared  to  have  been  detected. 


^Probability  of  deciding  U^  =0  while  Hj  is  true 
^Probability  of  deciding  U^  =  1  while  H^  is  true 
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Figure  1.1     Centralized  Detection. 
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Solution  of  the  CD  Problem 

The  solution  to  the  CD  problem  is  [6] 
a)  deterministic,  so  that  the  decision  rule  is  a  function  of  the  observations 


Y  :  Yj  xY^  x...xYj^  -  (0,1), 


(1.14) 


b)  a  likelihood  ratio  test. 


UJypy^,    y^) 


■{;: 


i^A(yj,y2..,yj^)  ^  t 
ifA(yj,y2,...,y^j)  <  t 


(1.15) 


where 


(1.16) 


c)  and  the  threshold  t  is  given  by 


t  =  C 


(1.17) 


2.  The  Decentralized  Detection  (DD)  Problem  with  Fusion 

Consider  the  structure  of  Figure  1.2  with  H  and  Y  being  as  before;  the 
decisions  L'^  Xj  '•••  '^^^  ^'n  ^^^  sent  to  a  fusion  center.  The  activity  of  the  fusion 
center  is  to  make  the  global  decision  U^  according  to  some  preset  fusion  rule. 


U^:U^xU2..xU^-(0,l). 


(1.18) 


In  the  DD  problem  with  fusion   it  is  required  to  design  local  decision  rules  UpU2,-- 
and  Uj^,  and  a  global  fusion  rule  (1.18)  so  as  to  minimize  the  expected  cost  E(J(Uq  ,H)) 


incurred  by  deciding  U^  =  i  when  H.   is  true. 


Choosing  an  AND  fusion  rule  apriori,  Teimey  and  Sandell  solved  tliis  problem 
for  N=2.  They  set  the  decision  rule  as  U^  =  U^  Vj  and  optimized  the  local  decision 
rules. 
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Fiaure  1.2    Decentralized  Detection  with  Fusion. 
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Solution  of  DD  Problem  >vith  Fusion 

The  solution  to  the  DD  problem  with  fusion  is 

1.  deterministic 

Yi:Yj-(0,l)  (1.19) 

and 

Y2:Y2-*(0,1)  (1.20) 

2.  a  likelihood  ratio  test  for  each  detector 

(    0,  if  A.  (V;x>    t- 

1.1,         ifA.(y..<ti 


where 


flv.  ,'H,  ) 
A.(y.)=      '^'      \      .  (1.22) 

3.     with  coupled  thresholds  t.  and  t^  given  by 

Pr(F,  /v,  ) 
t     =  C        '    '-   '^  '  (1.23) 

^  Pr(D2  /Vi  ) 


and 


Pr(F^  >%  ) 
Pr(Di  /y^  ) 


t2    =  C ^^^;  (1.24) 


where  Pr(F.  /y.  )  and  Pr(Dj  /y.  )  are  respectively  the  conditional  probability  of 
false  alarm  and  the  conditional  probabiUty  of  detection  of  the  i^  detector  given 
the  j^  detector's  observation. 
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Equations  (1.23  )  and  (1.24  )  are  two  coupled  functional  equations  in  t^  and  tj  .  For 
general  distributions,  a  functional  expression  for  each  of  them  in  terms  of  its  own 
observation  and  the  other  detector's  decision  is  impossible.  We  shall  consider  the 
complexity  of  these  decision  rules  later.  A  special  case  of  the  DD  problem  is  the  case 
of  conditionally  independent  sensor  observations,  i.e. 

f(yj/y2.H)=fl:yi/H)  (1.25) 

and 

fly2/yi.H)  =  fi:y2/H).  (1.26) 

In  this  case,  the  conditional  probabilities  in  (1.23  )  and  (1.24)  reduce  to 

t,    =  C ^  (1.27) 

1  p 

and 


t,    =  C-^  (1.28) 


^2 


Equations  (1.27)  and  (1.28)  are  two  coupled  algebraic  equations  in  the  form  of 

tj  =gi(t2)        and     t2  =g2(ti)  (1.29) 

since  P^.^  and  P^j  depend  on  t.  .    This  coupling  represents  cooperation  between  the  two 

sensors.    The  threshold  equations  are  necessary  conditions  for  optimality.  There  may 

be  several  local  minima;  each  must  be  checked  to  assure  the  global  minima.  The 

threshold  equations  are  strongly  coupled  for  general  cost  assumptions. 

Tenney  and  Sandell  came  to  the  following  conclusions: 

1.  Increasing  the  signal-to-noise  ratio  improves  the  performance  of  the  system. 
However  a  centralized  system  makes  more  efficient  use  of  the  increased 
information. 
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2.  As  the  imbalance  between  the  two  detectors  increases  the  performance 
improves.  If  the  signal-to-noise  ratio  of  one  of  the  detectors  goes  to  zero  then 
the  system  decision  is  that  of  the  other  detector.  This  is  equivalent  to  the 
performance  of  a  CD  system  of  the  same  signal-to-noise  ratio. 

The  case  of  conditionally  independent  observations  has  been  considered  by 
many  authors.  Sarma  and  Rao  [7]  extended  Tenney  and  Sandell's  results  to  the  case  of 
three  sensors.  They  assumed  a  majority  logic  fusion  rule  and  evaluated  the  threshold 
settings  for  some  specific  cases.  Chair  and  Varshney  [8]  considered  the  problem  of 
optimal  fusion  of  N  local  decisions  from  prespecified  local  decision  rules.  Their 
optimum  fusion  structure  is  a  weighted  sum  of  local  decisions  according  to  their 
reliabilities.  Reibman  and  Nolte  [9]  optimize  both  local  decision  rules  and  the  fusion 
rule  under  the  assumption  of  identical  local  decision  rules.  The  global  decision  is  then  k 
out  of  N.  They  optimize  the  local  decision  rule  for  each  k  ,k=  1,2,...,N,  then  pick  the 
value  of  k  corresponding  to  the  minimum  decision  cost. 

A  sub-class  of  the  DD  problem  with  fusion,  that  will  be  referrd  to  as  the 
"Second  Opinion"  problem,  is  the  fusion  of  one's  observation  with  another's  decisions. 
An  example  of  this  is  the  second  opinion  in  a  medical  examination,  or  even  asking  for 
legal  advise.  Ekchian  [10]  and  Ekchian  and  Tenney  [11]  consider  some  specific 
topologies  of  this  problem.  Each  decision  maker  has  to  make  his  decision  based  on  his 
own  observation  and  a  predecessor's  decisions.  All  the  decision  rules  are  likelihood 
ratio  tests  using  the  actual  data.  The  thresholds  are  determined  by  incoming 
communication  messages.  The  number  of  thresholds  at  each  decision  maker  grows 
exponentially  with  the  number  of  message  inputs.  Their  results  suggest  putting  the 
noisy  sensor  "up  stream"  in  the  detection  network. 

Papastavrou  and  Athans  [12]  also  consider  the  second  opinion  problem.They 
examine  the  structure  of  a  primarv'  decision  maker,  PDM,  and  a  secondary  decision 
maker,  SDM  (  a  consultant  ).  The  PDM  makes  his  decision  based  on  his  own 
observation  if  it  is  of  good  quality.  If  his  observation  is  noisy,  the  PDM  asks,  at  a 
communication  cost,  the  opinion  of  the  SDM.  Being  activated  by  the  request  of  the 
PDM,  the  SDM  sends  his  decision  to  the  PDM  or  ignores  the  request  if  his 
observation  is  noisy.  In  either  case  the  PDM  has  to  make  a  fmal  decision.  Again  the 
thresholds  are  coupled.  The  threshold  of  the  PDM  is  determined  by  the  message  of  the 
SDM. 

This  thesis  is  motivated  mainly  by  three  of  the  above  works  namely; 
1.     Bayesian  formulation  of  the  DD  problem  by  Tenney  and  Sandell  [5]. 
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2.  Extension  of  the  DD  problem  to  the  Distributed  Detection  Networks  by 
Ekchian.  [10] 

3.  Extention  of  the  DD  problem  to  the  case  of  correlated  sensor  observations  by 
^  Lauer  and  Sandell  [4]. 

C.      THE  COMPLEXITY  OF  THE  DD  PROBLEM 

We  saw  that  the  DD  problem  can  be  solved  optimally  for  conditionally 
independent  sensor  observations.  If  this  condition  does  not  hold  local  decisions  are  not 
likelihood  ratio  tests  with  constant  thresholds.  Tenney  and  Sandell  show  that  for 
conditionally  dependent  observations,  local  decision  rules  are  likelihood  ratio  tests  but 
with  data  dependent  thresholds  (see  e.g.  (1.23  )  and  (1.24  )).  These  two  equations  are 
coupled.  This  means  that  the  observation  of  one  sensor  is  necessary  for  the  other 
sensor's  decision,  which  contradicts  the  principle  of  decentralization.  In  terms  of  the 
terminology  of  the  Theory  of  Combinatorial  Complexity  [13],  Tsitisiklis  and  Athans 
[14]  show  that 

1.  The  DD  problem  with  independent  observations  is  a  polynomial  time  problem. 

2.  The  DD  problem  with  dependent  sensor  observations  in  its  simplest  form  is  a 
nondeterministic  polynomial  NP-complete.  This  means  that  exhaustive 
enumeration  is  necessary  to  find  the  optimum  local  decision  rules.  Optimality 
may  be  an  illusive  goal.  So,  suboptimal  solutions  must  be  sought. 

A  suboptimal  solution  to  the  problem  for  the  case  of  AND  fusion  was  considered 

by  Lauer  and  Sandell  [4].    They  considered  the  case  of  known  signals  in  correlated 

noise.    They  took  as  a  suboptimal  solution  local  decision  rules  which  are  likelihood 

tests  but  having  constant,  not  data  dependent,  thresholds  satisfying  the  necessary 

condition  of  optimality.   These  thresholds  are  given  by  the  implicit  equations: 

Pr(F.  /T,  ) 
A   (T,  )=  C  '      '  (1.30) 

^      ^  Pr(D2  /T^  ) 

and 


Pr(F,  /T^  ) 
A^  (T^  )=  C  — — ^^ — 2_L  .  (1.31) 

^     ^'  Pr(Di/T2)  ^       ^ 
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D.  CONTRIBUTIONS  OF  THIS  THESIS 

We  have  reviewed  the  complexity  of  the  DD  problem  and  its  current  status.  The 

research  reported  here  has  significantly  advanced  this  status  in  several  important  ways. 

Specifically  the  contributions  of  this  thesis  have  been  to  : 

1.     Answer  the  question  of  the  optimum  fusion  rule  at  the  fusion  center  for  the 
case  of  two  sensors. 

2o     Specify  the  exact  relation  between  the  performance  of  the  optimum  fusion  rule 
and  the  correlation  coefficient  between  sensor  observations. 

3.  Solve  the  the  second  opinion  decision  problem. 

4.  Solve  the  multi-level  DD  problem    with  fusion;  i.e.  detection  with  quantized 
sensor  data  for  the  known  signal  in  noise  case. 

5.  Introduce  the  minimum  risk  quantizer. 

6.  Grade  the  road  between  DD  detection  and  CD  detection. 

7.  Optimally  design  quantizers  for  minimum  mean  square  estimation. 

8.  Present  an  efficient  procedure  to  calculate  parameters  of  a  large  variety  of 
quantizers. 

E.  ORGANIZATION  OF  THE  THESIS 

The  thesis  is  organized  as  follows.  In  Chapter  II  we  consider  the  problem  of 
fusion  in  DD.  Optimum  detection  with  quantized  sensor  data  is  considered  in  Chapter 
III,  where  the  Quantized  Detection  algorithm,  QD,  is  introduced.  Numerical  e.xamples 
to  illustrate  the  algorithm  are  given  in  Chapter  IV.  Generalization  to  the  case  of  vector 
observations  is  presented  in  Chapter  V.  Optimum  regeneration  of  sensor  observations 
from  their  quantized  versions  and  another  sensor  observation  is  considered  in  Chapter 
VI.  A  summar}'  of  the  thesis,  conclusions  and  suggestions  for  future  research  are  given 
in  Chapter  VII.  Proofs  to  some  equations  and  FORTRAN  programs  to  calculate 
parameters  of  the  minimum  risk  and  the  minimum  distortion  quantizers  are  given  in 
the  appendices. 
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II.  OPTIMUM  FUSION  OF  LOCAL  DECISIONS 

In  this  chapter  the  important  question  of  the  optimum  fusion  rule  will  be 
answered.  The  relationship  of  the  optimum  fusion  policy  to  the  ratio  of  costs  and  the 
correlation  coefficient  between  observations  is  determined. 

A.      INTRODUCTION 

Distributed  Detection  with  fusion  is  a  two  level  optimization  problem.  The 
problem  can  be  formulated  in  the  following  three  ways: 

1.  Local  Decision  Optimization 

The  fu-st  way  is  to  select  the  fusion  rule  apriori  and  optimize  the  local  decision 
rules  accordingly.  Setting  the  activity  of  the  fusion  center  as  AND  fusion,  Tenney  and 
Sandell  [5]  derived  optimum  local  decision  rules  for  a  pair  of  spatially  separated 
detectors  with  conditionally  independent  observations.  They  prove  that  local  decision 
rules  are  simple  likelihood  ratio  tests  with  constant  thresholds.  The  thresholds  are  the 
solution  of  a  pair  of  coupled  algebraic  equations  that  correspond  to  the  global 
minimum  of  the  detection  cost  function.  They  also  show  that  for  the  case  of  correlated 
observations  local  decision  rules  are  likelihood  ratio  tests  but  with  data  dependent 
thresholds.  Functional  solution  of  the  threshold  equations  in  the  later  case  violates  the 
principle  of  decentralization.  Realizing  the  difficulty  of  the  problem  in  the  case  of 
correlated  observations,  Lauer  and  Sandell  [4]  designed  suboptimal  local  decisions  for 
AND  fusion.  Their  local  decision  rules  are  likelihood  ratio  tests  with  constant 
thresholds  satisfying  the  necessary  conditions  of  optimality.  Kovatana  [15]  considered 
AND  fusion  for  two  detectors.  Fefjar  [16]  compared  AND  to  OR  fusion  for  two 
detectors.  He  claimed  that  OR  is  better  than  AND.  Stearns  [17]  contradicts  Fefjar's 
results.  He  showed  by  an  example  that  OR  combining  is  better  for  higher  cost  of 
missing  the  target  while  AND  combining  is  better  for  higher  cost  of  false  alarms. 

2.  Fusion  Rule  Optimization 

In  the  second  formulation  of  the  problem,  local  decision  rules  are  set  apriori. 
Optimization  is  carried  out  with  respect  to  the  fusion  rules.  An  example  of  this 
situation  could  be  factory  built  sensors  that  cannot  be  adjusted.  Assuming  local 
threshold  settings  Chair  and  Varshney  [8]  prove  that  for  the  case  of  conditionally 
independent  sensor  observations,  the  optimum  fusion  rule  is  a  likelihood  ratio  test  that 
sums  local  decisions  weighted  according  to  their  reliability. 
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3.  Global  optimization  of  the  Local  decisions  and  the  Fusion  Rule 

The  third  formulation  involves  optimization  at  botli  levels.  Here  local 
decisions  are  optimized  for  every  possible  fusion  rule.  The  optimum  fusion  rule  is  the 
one  that  minimizes  cost. 

The  main  issue  of  this  chapter  is  the  global  optimization  of  the  DD  system  for 
general  correlated  observations.  First  we  will  state  the  main  results  for  the  case  of  N 
conditionally  independent  and  identically  distributed  sensor  observations.  Then,  the 
problem  of  fusing  two  local  decisions  of  sensors  with  correlated  observations  is 
considered. 

B.       GLOBAL  OPTIMIZATION  OF  DISTRIBUTED  DETECTION 

In  CD  all  sensor  observations  are  available  at  one  central  processor  for  detection. 
The  decision  rule  in  CD  is  a  likelihood  ratio  test  in  the  observations  yi,y2,---,yjs4-  It 
declares  H^  is  true  if  the  likelihood  ratio 

A  (yi,y2,...,y^. )  ^  C,  (2.1) 

otherwise  it  will  declare  Hq  to  be  true. 

In  DD  only  local  decisions  are  sent  to  the  central  processor  (  fusion  center).  The 
objective  of  the  fusion  center  is  to  mix  (  fuse  )  the  local  decisions  into  a  single  global 
decision  with  minimum  decision  cost.  So  given  the  local  decisions  the  observation 
space  of  the  fusion  center  consists  of  2"^  discrete  points.  The  activity  of  the  fusion 
center  is  to  divide  this  space  into  two  decision  regions  Zq  and  Zj.  The  decision  rule  of 
the  fusion  center  is  a  likelihood  ratio  test  [S.]  The  fusion  center  declares  H^  is  true  if 

A  {u^,u,,...,u^  )  >  C.  (2.2) 

otherwise  it  will  declare  that  Hg  is  true.  In  the  special  case  of  conditionally 
independent  and  identically  distributed  observations,  the  fusion  rule  is  a  k  out  of  N 
rule.  Reibman  and  Nolte  [9]  considered  this  problem.  Assuming  the  same  decision  rule 
for  every  detector  they  optimize  local  decisions  for  every  k,  k=  1,2,...,N  then  pick  the  k 
with  the  minimum  decision  cost. 

If  sensor  observations  are  not  conditionally  independent,  there  is  no  guarantee 
that  local  decisions  are  simple  likelihood  ratio  tests.  The  problem  turns  out  to  be  NP- 
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complete  which  needs  exhaustive  enumerations  to  find  the  optimum  decision  rules  [14.] 
Moreover  if  sensor  observations  are  not  identically  distributed,  there  are  as  many  as 
2      possible  fusion  rules  for  the  N  sensor  decisions.  Any  algorithm  that  goes  through 
the  entire  fusion  list  optimizing  local  decisions  will  be  impractical   for  N  ^  6. 
Our  approach  to  avoid  this  exhaustive  enumeration  is  the  following: 

1.  We  assume  that  local  decisions  are  likelihood  ratio  tests  with  constant 
thresholds.  Again  we  emphasize  that  this  assumption  is  valid  only  for 
conditionally  independent  observations,  there  is  no  guarantee  that  it  is  correct 
for  correlated  observations  [5].  So  the  constant  threshold  likelihood  ratio  test  is 
optimum  for  conditionally  independent  observations  and  perhaps  suboptimum 
for  correlated  observations.  However  the  solution  tends  to  the  optimum 
solution  as  the  correlation  coefficient  tends  to  zero  [4]. 

2.  Those  fusion  rules  which  agree  with  the  CD  solution  will  be  tested.  The  rest  of 
the  fusion  rules  will  be  disregarded.  The  meaning  of  this  will  be  made  clear  in 
the  following  example. 

Let  us  consider  the  case  of  two  sensors  {N   =  2  )  in  detail.    To  be  explicit, 

consider  detection  of  known  signals  in  gaussian  noise.  The  sensor  observations  are 

given  by: 

Hq:    y;  =  a,  i=l,2      .  (2.3) 

and 

Hj  :    y.,  =  a.  +  a,       i=  1,2.  (2.4) 

The  a.'s  are  positive  constants  and  N   =    [n^   n2]'  is  vector  of  zero    mean    with 
covariance 


=[;;] 


(2.5) 


where  p  is  given  by 

P  =  E  {  n^  n^  ).  (2.6) 


A  computer  that  spends  1  n  second  in  every  optimization  process,  will  spend 
40000  years  to  determine  the  optimum  fusion  rule,  for  N  =  6. 
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The  threshold  equation  of  the  CD  problem  is  given  by  [6], 

(aj  -p  a^  )yi  +(a2  -  P  a^  )y2  ={a^^  +a^^  -2p  a^  ^i^yi  +  d-p^  )log(C)  (2.7) 

which  is  a  straight  line  in  the  y,  y,  plane.    Figure  2.1  shows  decision  rules  based  only 
on  Dp  only  on  D2,  both  decision  rules  together,  and  the  decision  rule  of  CD. 

The  global  optimization  requires  optimizing  local  decision  rules  for  every  fusion 
rule  then  picking  the  fusion  rule  with  minimum  average  cost.  The  observation  space  of 
the  fusion  center  consists  of  four  discrete  points  (0,0),  (0,1),  (1,0),  (1,1).  Any  fusion 
rule  divides  this  space  into  two  decision  regions  Z^  and  Zq.  There  are  2"*  =  16  methods 
to  subdivide  four  points  into  two  groups.  Table  1  contains  a  list  of  those  fusion  rules. 
Some  special  cases  for  the  detection  problem  are  as  follows. 

1.  If  CjQ  -»  CO,  i.e.  the  cost  of  missing  the  target  is  extremely  high,  the  CD 
solution  assigns  all  the  observation  space  to  Zq.  The  fusion  center  can  perform 
the  same.  This  is  fusion  rule  one. 

2.  Similarly  if  Cq^  -♦  go,  the  fusion  center  will  always  decide  H^  this  is  fusion  rule 
two. 

3.  If  a2  =  p  a^  the  CD  will  decide  based  only  on  Vj.  So  will  the  fusion  center. 
This  is  fusion  rule  three.   This  can  only  happen  when  a^  ^  ^2- 

4.  If  aj  =  p  a.,,  the  CD  will  decide  based  on  y2.  This  is  fusion  rule  four.  This  can 
only  happen  when  a^  ^  a^ 

The  first  two  situations  represent    extreme  conditions  of  C.  The  next  two  conditions 

deal  with  specific  values  of  p.   We  also  distinguish  the  following  two  cases. 

Case  a 

-1  ^  p  :^  min(  a^  ,a2  )/max(  a^  ,a2  ). 

In  this  case  the  y^  and  y2  intersections  of  the  threshold  equation  (2.7  )  are  of  the  same 

sign. 

Case  b 

min(  a^  ,a2  )/max(  a^  ,a2  )  <  p  ^  1. 

In  this  case  the  y^  and  y2  intersections  of  the  threshold  equation  are  of  diiTerent  signs. 

We  shall  consider  these  intervals  of  p  when  we  study  the  effect  of  correlation  between 

sensor  observations. 

The  CD  threshold  in  the  y^  y2  plane  suggests  assigning  the  decision  point  (0,0)  to 

Zq  and  (1,1)  to  Zj.    The  fusion  rules  from  5  to  14  do  not  do  this.    They  either  assign 

(0,0)  to  Zj  or  assign  (1,1)  to  Zq  or  assign  (0,0)  and  (1,1)  to  the  same  decision  region. 
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Figure  2.1     Decision  Rules. 
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TABLE  1 
EXHAUSTIVE  FUSION  LIST  OF  TWO  DECISIONS 


Rule? 

Zo 

z. 

Comments 

1 

<t> 

(0.0).(0,I).(l,0).(!.l) 

Co,  -  =°  . 

2 

(0.0),(1.0).(0.1).(1,1) 

o 

Cjo-=° 

3 

(O.O).(l.O) 

(0.1).(1.1) 

^0-^2 

4 

(0,0).(0.1) 

(l.O).(l.I) 

Uo-L^I 

5 

(f).n 

(0.0).(I.0).(1.1) 

6 

(1.0) 

(o.(,)).(0.l),(l.i; 

7 

(1.1) 

(0.0).(1.0).(0.1) 

(O.D.d.O) 

(O.(i).(l.l) 

9 

1 

(o.i).(i.n                   (i>.(i).(i.o) 

10 

(l.Ol.(l.l) 

(0.01.(0,1) 

11 

(O.O).(l.l) 

(O.n.(i.o) 

12                (t).n.(i.i)).(i.i)                   (o.u) 

1 

13                       (().i').(U.I).(l.l) 

(1.0) 

14 

(n.i)).(l.(.)).(|.l)                          (i).l) 

,5 

1 

(o,i>).(().n4i.ii)                   (1,1) 

1 

.AND 

ir, 

(0.1))                      1      (O.I).(l.(i).(l.l) 

OR 
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We  will  not  consider  these  ten  fusion  rules  further.  The  remaining  two  decision  rules 
are  the  AND  fusion  and  the  OR  fusion.  Let  us  now  consider  optimizing  each  of 
them. 

1.  '  AND  '  Fusion 

In  AND  fusion  u   is  given  by: 


o 


u^  =  Uj  u^.  (2.8) 

The  individual  rules  are  given  by  assigning  yj  to  Z^  if 

yj  ^  Tj     ,i=l,2.  (2.9) 

Otherwise  thev  assien  it  to  Zr,. 

u 

The  probability  of  detection  Pj(AND)  and  probability  of  false  alarm  P^AND) 
of  the  fusion  center  are  given  by: 


P^  (AND)=  J^     J^    fTy^.y^  l^i  )  dy^  dy^  (2.10) 


and 


Pf.(AND)  =  J^    J^  ^yry2/Ho)dyidy2  (2.11) 


It  has  been  shown  in  Chapter  I  that,  to  within    positive  multiplicative  and 
additive  constants,  the  average  decision  cost  is  given  by 

R  =  1  +  C  Pj.-Pj.  (2.12) 

Substituting    for  P^  and  Pj.  in  (2.12)  from  (2.10)  and  (2.11)  expresses  R(AND)  as  a 
function  of  T^  and  T2.   The  necessary  conditions  for  opiimality  are 

d  Rid  T^  =  0  and  d  R!d  T^  =  0  ,  (2.13) 

which  can  be  written  in  the  forms: 
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.00  CO 

^2 


C  ij      ^'^vyi  /"o)  ^^2  =    Jj-       ^Tjj^  /Hi  )  dy^  (2.14) 


and 


.00  '  00 


C    L      flYpT^  /Hq)  dy^  =    r        flVpT^  /H^  )  dy^.  (2.15) 

Applying  Bayes  rule  and  rearranging  terms,  one  can  write  (2.14)  and  (2.15)  as  follows: 


J-r      fly2/Ti  .Hq)  dy. 
\  (T^  )  =  C  -:} (2.16) 


00 
2 


J -I-      flyi/Ti  ,Hj)  dy^ 


and 


A2(T2)=  C-— J .  (2.17) 

^        ^  CO 


M 

C  ^yi''T2 'Hi)  ^yi 

To  insure  minima  the  Hessian  matrix  of  R  with  respect  to  T^  and  T2 


(2.18) 


must  be  positive  definite.  Optimum  threshold  settings  T.  and  T2  are  the  solution  of 
(2.16)  and  (2.17)  that  corresponds  to  the  global  minima,  so  all  possible  solutions  of 
(2.16)  and  (2.17)  must  be  tried.  The  coupling  between  (2.16)  and  (2.17)  to  determine 
the  thresholds  represents  the  cooperation  that  can  occur  between  the  two  local 
detectors  to  minimize  the  overall  decision  cost. 
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2.  'OR'  Fusion 

The  decision  of  the  OR  fusion  is  given  by 

Uq  =  u^  +  u^  -  u^  u^.  (2.20) 

The  probability  of  detection  P^  (OR)  and  probability  of  false  alarm  P^  (OR)  are  given 
by 

P,  (OR)=  1  -  f  1       f  2   ^y^^y^  /H,  )  dy,  dy^  (2.21) 


,00        •'.00 


and 


Pf  (0R)=  1  -  J^l      Jl  flvj.y,  IH,  )  dy,  dy,  (2.22) 


.00      •'.oo 


while  the  necessary  conditions  for  optimality  are 


L     fl:y2.'T,  ,Ho)  dy. 


too 


A  (T^  )  =  C  — (2.23) 

U    ^y2/Ti,H,)dy2 


and 


0   f^yi''T2  .Ho)  dy, 
A(T2)=  C— .  (2.24) 

U   «yi/T2  .H,)  dy, 


ioo 


Again  the  Hessian  matrix  must  be  positive  defmite. 
3.  Solution  of  the  Nonlinear  Tlireshold  Equations 

The  pair  of  coupled  equations  (2.16),  (2.17)  for  the  AND  fusion  and  (2.23) 
and  (2.24)  for  the  OR  fusion  can  be  solved  using  Max's  technique  [18].   The  technique 
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is  summarized  as  follow:  pick  a  value  of  T^  and  calculate  T^  from  (2.16)  or  (2.23  ).  If 
the  calculated  value  of  Tj  does  not  agree  with  that  value  calculated  from  (2.17)  or 
(2.24)  then  Tj  must  be  chosen  again.  This  approach  is  time  consuming.  Another 
approach  is  the  method  of  successive  substitution  [19].  We  first  put  the  two  equations 
in  the  form 

T,      =  G(T,  ,T,  )  ,  T,     =    F(T,         ,T,  )  (2.25) 

^k+1         V  H        ^k+l         lk+1     ^k  ^ 

then  start  with  a  reasonable  guess  for  (T^  )q  and  (T2  )q.  A  suitable  initial  guess  is  the 
locally  optimum  solutions,  i.e.  the  thresholds  that  would  optimize  the  detection  if  each 
sensor  works  alone.  These  will  be  denoted  by  T.j^  and  T2JQ.  For  known  signals  in 
gaussian  noise  these  are 

(Tj)o  =ai/2  +log(C)/a..  (2.26) 

4.  Numerical  Results 

We  have  solved  the  threshold  equations  for  both  fusion  rules  for  a^  =1.7  and 
a,  =  2.3  for  several  values  of  p  and  C. 

To  compare  AND  and  OR  fusion,  define  K  as  the  ratio  of  the  AND  cost  to 
the  OR  cost. 

1  +  C  P.  (AND)-  P.  (AND) 

K    =     ^- ^ (2.27) 

1  +  CPr(OR)-  p.  (OR) 


r  ^  '        a  ^ 

CPf(OR)-P^(OR) 


We  have  also   computed  the   Receiver  Operating  Characteristic"*  (ROC)  curves   of 
classical  communication  theor>'  [20]  for  each  fusion  rule. 

Figure  2.2  shows  the  ratio  K  as  a  function  of  C  for  p  =  0,  0.2  ,  0.4.  The 
figure  shows  that  AND  fusion  is  optimum  for  C  ^  1  and  OR  fusion  is  optimum  for 
lower  values  of  C.  The  same  is  clear  from  Figure  2.3;  ROC  curves  of  AND  fusion  are 
above  those  of  OR  fusion  for  C  ^  1  and  lower  otherwise.  The  performance  difference 
becomes  smaller  as  the  correlation  coefficient  increases.  Also  the  figures  show  that  the 
performance  degrades  for  both  fusion  rules  as  p  tends  to  one.  This  is  in  sharp  contrast 
to  CD  which  has  perfect  detection  for  p=  I. 


'^P^  as  a  function  of  P^ 
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Figure  2.2     Ratio  of  Costs  of  AND  and  OR  Fusion  Rules 
a,  =  1.7,  a,  =2.3. 
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Figure  2.3     ROC  Curves  of  AND  and  OR  Fusion  Rules 
a^  =  1.7,a2  =2.3. 
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The  same  effects  can  be  concluded  from  Figure  2.4  and  Figure  2.5.  Figure  2.4 
shows  the  ratio  K  as  a  function  of  C  for  a^  =  1  and  aj  =  2  and  for  p  =  0,  0.25,  0.5. 
Figure  2.5  shows  the  ROC  curves  for  both  fusion  rules  for  the  same  case.  The  figure 
shows  that  AND  fusion  is  optimum  for  C  ^  1  and  OR  fusion  is  optimum  for  lower 
values  of  C. 

C.      THE  EFFECT  OF  CORRELATION  BETWEEN  SENSOR  OBSERVATIONS 

So  far  we  have  answered  the  question  of  the  optimum  fusion  rule.  For  C  ^  1 
AND  fusion  is  optimum.  Let  us  now  examine  the  effect  of  the  correlation  coefficient  p 
on  the  performance  of  AND  fusion  for  C  ^  1  (its  range  of  superiority).  We  assume 
without  loss  of  generality  that  a2  is  greater  than  a^.  The  two  necessary  conditions  for 
optimality  of  AND  fusion  are  (2.16)  and  (2.17).  For  the  problem  of  known  signal  in 
gaussian  noise  these  can  be  written  as: 

Ai  (Tj  )  =  C X  r-r TT"  ^2.28) 


I    v(iy)      / 


and 


Is/(1-P-)        / 

X    V  d-p')      / 


erfc 
A2  (T2  )  =  C >  :  ^    '    ;  _^ TTT-  •  (2.29) 


Notice  that  C  appears  only  as  a  multiplicative  constant  in  the  two  equations.  The  role 
of  p  is  not  that  obvious.  Examining  the  two  equations  leads  to  the  following  insights 
about  the  role  of  p: 

1.  T^  =-co  and  T2  =T2Jq  is  a  solution.  This  corresponds  to  the  decision  rule  of 
D,. 

2.  T-,  =-co  and  T^  ~T|]o  is  a  solution.  This  corresponds  to  the  decision  rule  of 
D-. 

3.  If  a2  is  greater  than  a^  we  expect  the  performance  of  D2  alone  to  be  better  than 
that  of  Dj  alone  and  that  of  the  selfish  decision  rule  in  which  each  detector  tries 
to  minimize  its  own  detection  cost,  not  the  system  decision  cost,  by  using  T^j^, 

"^210- 
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Figure  2.4     Ratio  of  Costs  of  AND  and  OR  Fusion  Rules 

a^  =1,  a^  =2. 
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We  now  prove  three  lemmas  concerning  these  equations. 
1.     Lemma  1. 

For  p  ^  a^  /aj, 


and 


T    <  T 


T    <  T 
^2  -    4lo 


where  T.^^  is  the  optimum  threshold  of  the  ith  detector  operating  alone. 

Proof: 

Since  the  argument  of  the  complement  of  the  error  function  in  each 
denominator  is  less  than  the  argument  in  the  corresponding  numerator,  the 
fraction  is  always  less  than  one.  This  implies  that 

Aj(T)  <  C,i=U2. 

Lemma  2 


For  p  =  a^  /a^,  the  only  possible  solution  of  (2.28)  and  (2.29)  is: 

T    =T 
and 

T,  =.=0. 

Proof: 

For  p  =  aj  1^2  equation  (2.29)  becomes 

A2(T2)=C  =A2(T.,J.  (2.30) 

The  corresponding  value  of  T,  is  T,  =-oo. 
3.     Lemma  3 

For  p  >  a^  /a->  the  optimum  solution  for  Tj  and  T^  is: 

and 


Tj  =-so 


T    =  T 

This  means  that  the  decision  of  the  optimum  AND  fusion  is  that  of  D2. 

Proof: 

Recall  that  the  CD  threshold  line  divides  the  observation  space  into  two 
decision  regions.  For  positive  signals  the  following  inequality  is  satisfied  in  the 
resion  to  the  right  of  the  CD  line: 

Cf(yi,y2/Ho)<  f(y^  .y^ /H^  )  .  ■  (2.31) 
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The  converse  of  this  inequality  is  true  in  the  left  region.  The  decision  region  Z^ 
of  any  other  decision  rule  contains  areas  from  the  right  and  from  the  left  of  the 
CD  line.  Areas  to  the  right  will  have  a  negative  contribution  to  the  decision 
cost  while  areas  to  the  left  will  have  positive  contributions.  Now  assume  that  T^ 
and  Jj  ,  where  both  are  finite,  satisfy  the  necessary  condition  (2.28)  and  (2.29). 
We  shall  prove  that  they  cannot  correspond  to  the  optimum  solution.  The 
finite  point  (  T,  ^"2  )  either  lies  to  the  left  or  to  the  right  of  the  CD  threshold 
line  as  shown  in  Figure  2.6  a  and  b  respectively.  In  Figure  2.6  a  the  intersection 
of  the  CD  line  with  the  line  y^  =  Tj  is  a  better  solution  since  it  excludes  an 
area  in  which  C  f^y^  .yj  /Hg  )  is  greater  than  f{y^  ,  y^,  /^i  )•  ^  better  solution 
than  this  has  the  same  T2  but  with  T,  =  -co  since  the  added  area  has  negative 
contribution  to  the  cost.  In  Figure  2.6  b,  T^  =-co  and  Y^  is  a  better  solution 
than  Tj  and  l^,  since  the  added  area  has  a  negative  contribution  to  the  cost. 
In  both  cases  T^  =  -00  is  the  optimum  solution  and  the  corresponding  optimum 
value  of  T2  is  T,j^. 

As  a  result  of  the  above  three  lemmas  it  is  clear  that 

1.  Any  solution  of  the  necessar>' conditions  must  satisfy 

T    <  T 
^2  -    4lo 

2.  The  performance  of  the  AND  fusion  saturates  to  that  of  D,  alone  for  p  ^ 
aj/a2.  We  might  recall  that  the  threshold  line  of  the  CD  system  changes  slope 
at  that  value  of  p.  We  will  refer  to  this  value  of  p  by  p^^..  This  result  is  in 
contradiction  with  Lauer  and  Sandell's  results  [4]  which  shows  performance 
continuing  to  degrade  with  increasing  p  for 

P   ^   Per- 
Limiting  behavior  for    =  p-l. 

For  p  =-1  the  joint  probability  density  function  n[y^  ,y2  /Hq  )  has  values  only  on  the 

line  y^   =  -y^.    So  any  threshold  values  T^  and  T^  such  that  Tj    =  -T2  will  produce 

AND  fusion  with  zero  probability  of  false  alarm.  This  can  be  visualized  from  Figure 

2.7  .  Consequently,  P^  will  be  given  by 


Pj  =  0.5  erfc  <  T,  -a 


\    "^2  -H    \  -^-^  ^^^^  f  "^2  "^^1  V  (2-^2) 


Maximizing  P^  with  respect  to  T^  yields 

T2  =(a2-a^)/2.  (2.33) 

For  the  special  case  of  equal  SNR  sensors,  T-,  =  0. 
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Figure  2.6    Optimum  T^  and  T2  for  p  ^  a^  /a^. 
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D.      NUMERICAL  RESULTS 

The  average  decision  costs  vs  p  for  a,  =  32=  2,  and  C=  1  are  shown  in  Figure  2.8. 
Threshold  values  Tj  and  T2  vs  p  for  the  same  case  are  shown  in  Figure  2.9.  Figures 
2.10  and  2.11  show  the  same  for  C=  10. 

These  four  figures  for  the  case  of  equal  signal-to-noise  ratio  show  that  the  two 
detectors  cooperate  with  each  other  using  the  same  decision  rule  (  equal  thresholds  ). 
Their  threshold  is  an  increasing  function  of  p.  The  limit  of  this  threshold  as  p  ->  -1  is 
zero.  This  behavior  agrees  with  (2.33).  The  limit  of  the  threshold  as  p  -♦  1  is  Tj^.  This 
is  because  for  p  -^  1  the  two  systems  have  identical  observations. 

The  detection  cost  curves  show  that  the  cost  is  an  increasing  function  of  p.  The 
curve  of  the  AND  fusion  has  the  same  shape  as  the  curve  of  the  CD  system.  Both 
systems  attain  their  best  performance  at  p  =  -l.  They  have  the  same  worst  performance 
for  p=  1. 

Figures  2.12  and  2.13  represent  the  case  of  unequal  SNR  sensors  for  C=l. 
Figures   2.14  and  2.15  show  the  same  for  C=  10. 
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Figure  2.10    Average  Decision  Costs  for  Equal  SNRs,C=  10. 
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Figure  2. 1 1     Threshold  Value  for  Equal  SNRs,C  =  10. 
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Figure  2.12    Average  Decision  Costs  for  Unequal  SNRs,C=  1. 
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Figure  2.13    Threshold  Value  for  Unequal  SNRs,C=  1. 
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Figure  2.15    Tlireshold  Value  for  Unequal  SNRs,C=  10. 
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These  four  figures  for  the  case  of  unbalanced  sensors  show  that,  the  two 
detectors  cooperate  using  different  thresholds.  The  threshold  of  the  higher  signal-to- 
noise  detector  is  an  increasing  function  of  p  while  the  other  threshold  is  a  decreasing 
function  of  p. 

The  cost  curves  show  that  the  fusion  rule  has  its  best  performance  at  p  =-1. 
Both  DD  and  CD  have  their  worst  performance  at  p=  p^.  For  p  >  p^  the 
performance  of  the  optimum  fusion  rule  is  the  same  as  the  detector  of  the  higher 
signal-to-noise  ratio.  Recall  that  CD  system  has  perfect  detection  for  p  =  1  when  the 
SNR's  are  unequal.  As  C  increases  the  average  cost  of  each  system  increases.  This  can 
be  explained  from  the  expression  for  R  in  which  the  probability  of  false  alarm  is 
weighted  by  C. 

E.       DISCUSSION  AND  CONCLUSIONS 

We  have  shown  that  the  optimum  fusion  rule  is  determined  by  the  ratio  of  costs 
and  the  apriori  probabilities.  For  equal  error  costs  AND  and  OR  fusion  rules  are 
equivalent.  This  is  not  surprising  since  each  system  turns  out  to  be  the  minimum 
probability  of  error  detector;  thresholds  are  adjusted  such  that  1  -Pj=  Pp  It  might  also 
be  noted  that  the  optimality  of  the  fusion  rule  is  independent  of  the  correlation 
coefficient  and  the  signal-to-noise  ratio  in  tliis  case.  We  also  note  that  the  detection 
cost  of  the  optimum  fusion  rule  has  its  minimum  value  at  p  =  -l.  It  has  its  maximum 
value  at  p  =  aj/a2.  The  performance  saturates  at  the  cost  of  decision  of  the  detector  of 
higher  SNR.  In  the  interval  (p  e  [3.^/3.2,1]),  the  optimum  fusion  rule  ignores  the 
decision  of  the  detector  of  lower  SNR.  As  a  good  dynamical  example  that  agrees  with 
this  result  is  the  switched  diversity  combiner  [21]  in  fading  environments  and  its 
centralized  counterpart,  the  maximum  ratio  diversity  combiner  [22].  Recall  that  for 
unequal  SNRs  the  performance  of  the  CD  system  improves  in  this  interval  and  has 
perfect  detection  for  p  =1.  Also  it  is  important  to  note  that  the  optimum  thresholds 
of  the  individual  observers  are  not  the  same  as  if  they  were  operating  independently, 
but  must  be  determined  by  simultaneous  solution  of  two  coupled  nonlinear  equations. 
This  represents  the  cooperation  between  the  two  detectors  to  work  as  a  team.  Lastly 
the  performance  difference  between  CD  and  DD  is  due  to  the  information  loss  in  local 
data  processing.  However  DD  has  fewer  requirements  on  the  communication  channel 
in  contrast  to  CD  which  requires  infinite  bandwidth.  A  compromise  between  these  two 
extremes  is  to  allow  more  information  than  just  decisions  to  be  sent  to  the  fusion 
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center.  This  is  the  concept  behind  the  Quantized  Detection  algorithm  considered  in  the 
following  two  chapters. 
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III.  DETECTION  USING  QUANTIZED  SENSOR  OBSERVATIONS 

A.  INTRODUCTION 

So  far  detection  with  sensor  observations  has  been  described  using  two  methods. 
In  the  first  metliod  all  sensor  observations  are  sent  to  some  central  processor  which 
makes  a  decision  based  on  a  likelihood  ratio  test.  In  the  second  method  only  local 
decisions  are  sent  to  the  central  processor  which  fuses  these  decisions  into  a  global 
decision.  While  the  first  method  is  very  easy  to  design  it  requires  in  principle  infinite 
bandwidth  communication  channels.  The  second  method  requires  only  one  information 
bit  per  detection.  Detection  with  quantized  sensor  observations  will  be  introduced  in 
this  chapter.  The  main  goal  of  the  chapter  is  to  grade  the  road  from  the  DD  problem 
to  the  CD  problem.  It  will  be  referred  to  by  Quantized  Detection,  QD.  The 
performance  improvement  of  the  DD  problem  will  be  traced  as  the  amount  of 
information  delivered  to  the  fusion  center  increases. 

First  let  us  consider  the  problem  of  the  Primary  Decision  Maker  (PDM)  and  its 
quantized  second  opinion  (consultant).  We  will  prove  thi-ee  theorems  concerning  the 
decision  rule  of  the  PDM.  Then  fusion  of  two  quantized  observations  of  an  arbitrary 
number  of  levels  will  be  considered.  As  a  special  case,  fusion  of  two  sensor 
observations,  one  quantized  to  N  levels  and  the  other  to  N  +'  1  levels,  will  be  proven 
equivalent  to  the  PDM  and  an  N-level  quantizer.  Comparison  between  different 
configurations  will  follow. 

B.  TEAM  DECISION  OF  A  PRIMARY  DECISION  MAKER  AND  A  SECOND 
OPINION  QUANTIZER. 

1.  Formulation  of  the  PDM  Problem 

Consider  the  structure  of  Figure  3.1  in  which  yj  is  quantized  into  y,     by  the 
quantization  rule  a  of  N  levels. 

a:Y^-Y^q.  (3.1) 

The  primary  decision  maker  will  make  his  decision  u  ,  about  the  phenomena  H  based 
on  its  own  observation  y^  and  the  quantized  observation  y.  . 
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Figure  3.1     Configuration  A,  The  Primary  Decision 
Maker  and  its  Quantized  Consultant. 
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The  problem  of  the  PDiM  is 

1.  to  design  the  quantization  rule  a  i.e.  to  specify  the  set  of  N  points 

-00  =  Xj  <  X2  ^  ...  <  Xj^  <  00 
that  defines  the  quantizer  intervals,  and 

2.  to  design  the  decision  rule  Yj 


in  order  to  minimize  the  decision  cost. 
2.  Problem  Analysis 

Our  approach  is  as  follows.  We  first  design  the  optimum  Bayes  decision  rule 
given  a  set  of  quantizer  parameters.  Next,  the  average  cost  is  expressed  as  a  function  of 
these  parameters.   We  then  minimize  the  average  cost  with  respect  to  them. 

a.    The  Optimum  PDM  Given  Some  Quantization  Rule    a 

We    have    shown    in    Chapter    I    that,    to    within    an    additive    and    a 
multiplicative  positive  constant  the  average  cost  is  given  by  [6] 

R  =  CPf-P^  (3.3) 

where  C  is  the  ratio  of  error  costs  and  Pj.  and  P^  are  the  probability  of  false  alarm  and 
probability  of  detection  respectively.  The  PDM  receives  a  quantized  level  y^  =  Q-  • 
He  will  make  his  decision  on  the  basis  of  his  own  observation  v.  and  v,     ,    The 

"2  '  Iq 

performance  of  the  the  primary  decision  maker,  given  some  quantization  rule  a,  is 

given  by  the  following  lemma. 

Lemma  3.1 

The  probability  of  detection  and  probability  of  false  alarm  of  the  Primary  Decision 

Maker  are  given  by: 

and 
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Pf  =  i  jj^ '  L  ^  pv^i  /Ho )  dyi  dy2  (3.5) 

where  Zy  is  the  decision  region  Z^  given  that  y^e[X.,X.^  j]. 

Proof: 

The  proof  is  given  in  Appendix  (A). 

The  decision  rule  of  the  Primary  Decision  Maker  is  given  by  Theorem  3.1. 
Theorem  3. 1 

Given  y^    and  y^  the  decision  rule  of  the  Primary  Decision  Maker  of  Figure  3.1  is 
1.     deterministic 


^2  •  ^iq  X  Y2  -*  (  0  ,  1  )  (3.6) 


2.  a  likelihood  ratio  test 

r    1     ifA{y, )  ^  0;(y, ) 

u^  =  J  '^  J  ^2^  ,j=1.2 N  (3.7) 

^      \    0     ifA(yO  <    0.(y,) 

whereA(y2)  =  fIy2/Hj)/fi:y2/Ho) 

3.  the  threshold  function  0.  {y^  )  is  given  by 

J^^^^yi/y2'Ho)d>'i 

0j  (y2  )  =     C^ ,j=  1.2,...N.  (3.8) 

J^^^^yi/y2'Hi)ciyi 

Proof 

We  first  insert  (3.4)  and  (3.5)  into  (3.3).  Each  term  of  the  detection  cost  (3.3)  is  then 
given  by 

R]  =  L  ^  I V  ^  ^  ^  ^yi  'yi  /Ho )  ■  ^yi  '^i  /"i )  ]  ^yi  ^^i  (^.9) 

To  make  R.  in  (3.9  )  negative  an  optimum  decision  rule  assigns  y^  to  Z^  if 
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C  I^^'"^yi  '^2  /"o  )  ^^1  ■  i^^^  ^yi  '^2  /"i  )  dyi  ^  0    ,i=  1,2,...N  (3.10) 

•X.  X. 

otherwise  it  will  assign  ^^  to  Zq  . 

Applying  Bayes  rule  and  rearranging  terms,  decision  rule  (3.10)  can  be 
written  as 

jJ^Wy2'Ho)dyi 

A  (y^  )  >  C  ^ ,  i=  1,2,...N  (3.11) 

which  completes  the  proof. 

h.   Optimum  Quantization  of     Y  , 

According  to  Tlieorem  3.1,  the  decision  rule  of  the  PDM  is  a  likelihood 
ratio  test  with  data  dependent  threshold.  The  threshold  depends  on  the  choice  of  X.'s. 
To  find  an  optimum  solution  for  the  X.  's  is  not  any  easier  than  that  of  the  DD 
problem.  Recall  that  for  the  DD  problem  optimum  solutions  are  possible  only  for  the 
case  of  conditionally  independent  observations.  Only  suboptimal  solutions  are  possible 
for  the  case  of  correlated  observations.  We  wUl  not  expect  more  for  the  QD  problem. 
Let  us  consider  each  case  separately. 

3.  Conditionally  Independent  Observations 

Under  the  assumption  of  conditionally  independent  observations,  i.e. 

fryi;y2,H)  =  f{y^/ H  )  (3.12) 

the  decision  rule  of  the  Primary  Decision  Maker  can  be  simplified.  This  decision  rule  is 
given  by  the  following  corollary  of  Theorem  3.1. 
Corollary  1 

Assuming  conditionally  independent  sensor  observations,  and  given  y^     and  Vj,  the 
decision  rule  of  the  Primarv'  Decision  Maker  of  Figure  3.1  is 
1.     deterministic 


Y2  =  '^lq^Y2-*(0,l)  (3.13) 
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2.     a  likelihood  ratio  test 


r    1      ifA(y,  )  ^  0- 
u    =  ^  ^  ^      ,j=l,2,...,N  (3.14) 

^       \    0     ifA(y2)<    0.     '^      "    ' 

whereA(y2)  =  fIy2/Hi)/fIy2/Ho) 
3.     the  threshold  0.  is  given  by 

jJ'"VHo)dy, 
0j  =  C ^ '■ ,j=l,2,...N.  (3.15) 

Proof 

By  applying  condition  (3.12)  in  the  threshold  equation  (3.8  )  one  obtains  (3.15)  which 
completes  the  proof. 

Let  us  denote  the  conditional  probability  of  detection  and  the  conditional 
probability  of  false  alarm  of  the  PDM  given  that  the  j^  quantization  level  of  y^  is 
received  by  P^.  and  P^-.    Let  T.  be  the  set  of  all  points  y2  for  which 

•      A2(y2)^0j  (3.16) 

Then  P ,.  and  P^  can  be  written  as 
dj  fj 

Pdi  =  I  ,p  fi>2 ''Hi )  <iy2  (3-17) 


J 


and 


J 
Equations  (3.4)  and  (3.5)  are  now  given  by 
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and 


Pf  =;!,  ^'*ftyi  /Hq  )  dy,  I^«y2  /Hq  )  %  •  (3.20) 


Substituting  (3.19)  and  (3.20)  in  (3.3),  then  differentiating   R  with  respect  to  X. 
,{j  =  2,3,...N}  will  yield  necessary  conditions  for  optimality  of  the  set  of  N  equations. 


c  [J™«\  ^2  /Ho  )<iy2  -L  ((\  .y2  -'Ho  )dy2 1- 

k  k-1 

[Jxp^^k'y2/"l)^y2-.^    fTX^,y2/Hi)dy2]  =  0   ,k  =  2,3 N.  (3.21) 


Applying  Bayes  rule  and  rearranging  terms,  (3.21  )  can  be  written  in  the  following  way. 

i^,  Ryj'Ho)  dy2  - 1  ,p  Ryj'Ho)  <iy2 

A{X^)  =  C-r-!^ -. — ^ ,k=2,3 N  (3.22) 

'  J.,,    Ry2'H,)dy,-|        lbVH,)<ly2 

*k  ^k-1 

The  set  of  N-1  necessary  conditions  (3.22)  are  general  for  any  statistics  of  y^  .   For  the 
special  case  when  A  (y^  )  is  monotonic  in  y2  ,  let  T.  be  the  value  of  y^  for  which 

e.  =A2(Tp   ,j=l,2,...,N.  (3.23) 

So  T.  is  given  bv 


J      ^ 


A  (T.  )=  C  -^ ,]•=  1,2,...N  (3.24) 

J^i  +  VH,)dy^ 
j 
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For  this  case  ofmonotonic  A  (yj  )  the  set  of  necessary  conditions  for  optimality  (3.22) 
can  be  written  as 

A  (X.  )=  C ^ ^ ,k=  2,3,..«,N  (3.25) 

''•00  CO 

J_    fl:y2/Hi)  dy^  -  L     r{y,lU,)dy, 
Equivalently  we  can  write  (3.25)  in  the  form 


J/''fiy2''Ho)dy2 

A  (Xj^ )  =  C  -:^ ,k=  2,3,...,N  .  (3.26) 

J-J'-ifIy,/H^)dy, 
^k 

P^  and  Pj.  in  this  case  are  given  by 

Pd  =_t    L'^yi  'Hi  )  dy^    L  f(y2  /Hi  )  dy^  (3.27) 

1  ^^i        A.-  1  • 


and 

Pf  =  1,    if'^'  ^yi  'Ho  )  dy,  Lfly.  /Hg  )  dy^  (3.28) 

l-l      A.  i. 

Equations  (3.24)  and  (3.26)  are  only  necessary  conditions  for  optimality  for 

monotonic    likelihood  ratio.     They   correspond   to   minima     if  the    Hessian   matrix 

[.d^RldX.dX.]  is  positive  definite.  All  solutions  must  be  checked  for  the  global  minima. 

4.  Solution  of  the   Primary  Decision  Maker  Problem  with  Independent  Sensor 
Observations  and  Monotonic  Likelihood  Ratio 

The  following  theorem  summarizes  the  above   solution  of  the  PDM   with 

independent  sensor  observations  and  monotonic  likeliliood  ratio. 
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Theorem  3.2 

The  decision  rule  of  the  Primary  Decision  Maker  with  a  Quantized  Consultant  (for 
independent  sensor  observations  and  monotonic  likelihood  ratio)  is; 
1.     deterministic 


Y2  •  Yiq  X  Yj  -»  (  0  ,  1  )  (3.29) 


2.  a  likelihood  ratio  test 

r    1      if  A(y,  )  >  0. 
u    =  ■{  ^  J    ,j=l,2,...,N  (3.30) 

«       \    0     ifA(y2)  <    a  '      "    ' 

whereA(y2)=fly2/Hj)/fIy2/Ho) 

3.  the  threshold  function  0  (y2  )  is  given  by 

Oj    =  C  ^-i ,j=l,2,...N.  (3.31) 

'    J 

The  optimum  set  of  quantizer  interval  end  points  must  satisfy  the  set  (3.26  ),  where 
Tj^'s  are  given  by  (3.24).  All  possible  solutions  must  be  checked  for  the  global 
minimum  cost. 

5.  The  Case  of  Correlated  Observations 

We  now  move  to  a  more  realistic  situation  by  removing  the  condition  of 
independent  sensor  observations.  In  many  radar  and  sonar  problems  noise  in  nearby 
sensors  is  likely  to  be  correlated.  As  we  mentioned  before  the  decision  rules  (3.11)  are 
likelihood  ratio  tests  with  data  dependent  thresholds.  It  is  impossible  to  come  with 
their  optimum  functional  expressions  [4.]  A  suboptimal  solution  for  the  case  of 
correlated  observations  is  to  use  likelihood  ratio  tests  with  constant  thresholds  as  local 
decision  rules.  These  constant  thresholds  for  y2  are  the  values  of  y2  for  which  the 
inequality  (3.11  )  is  an  equality,  i.e.; 
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A  (T.  )  =  C  4r X=  1.2,...,N  .  (3.32) 


In  terms  of  these  thresholds  Tj^  's  and  the  quantizer  points  Xj^  's  one  can  write 
expressions  for  the  probability  of  detection  and  the  probability  of  false  alarm  in  the 
form  of  (3.4)  and  (3.5).  Substituting  for  P^  and  P^  in  (3.3)  and  differentiating  R  with 
respect  to  Xj^  for  k=2,3,...N  yields  the  following  set  of  necessary  conditions  for  the 
case  of  monotonic  A2  (72  ) ' 

A  (X,  )  =  C  -^ ,k=  2,3,...,N.  (3.33) 

T,, 


J^''-'«>VX,,Hi)  dy^ 


The  set  of  equations  in  (3.32)  and  (3.33)  constitute  2N-1  equations  that  specify  the 
quantizer  interval  end  points  (Xj^  }  for  yj  and  the  thresholds  (Tj^}  for  y->  . 

C.       TEAM  DECISION  OF  TWO  QUANTIZERS  AND  A  FUSION  CENTER 

In  this  section  we  will  consider  the  problem  of  making  a  global  decision  based  on 
two  quantized  observations. 

1.  Formulation  of  the  QD  problem 

For   the   structure    of  Figure    3.2,   y^    is    quantized   into   N    levels    by   the 
quantization  rule  a, 

a^  :  Y^  -^  Yj^  (3.34) 

and  y2  is  quantized  into  M  levels  by  the  quantization  rule  02 

«2  =  Y2->Y2q.  (3.35) 

The  quantized  values  y^  and  y^  are  sent  to  the  fusion  center  which  must  decide  which 
state  of  the  phenomena  is  true.  It  is  required  to  design  the  quantization  rules  a^  and 
a2  and  the  decision  rule  y 
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Y  •  Yiq  X  Y2q  -*  (0,1)  (3.36) 

to  minimize  the  global  cost. 

2.  Problem  Analysis  and  the  QD  Algorithm 

The  observation  space  of  the  fusion  center  contains  NM  points  to  be  divided 
into  two  decision  regions.  Since  there  are  as  many  as  2"^^  fusion  methods,  checking  all 
of  them  will  consume  a  very  long  time  even  for  small  values  of  N  and  M.  A  suboptimal 
solution  is  to  approximate  the  threshold  equation  of  the  corresponding  CD  problem  by 
a  piecewise  curve  in  the  y^  y2  plane.  This  is  illustrated  in  Figure  3.3. 

The  figure  shows  a  schematic  diagram  of  a  CD  threshold  curve  and  its 
staircase  approximation.  The  approximate  curve  consists  of  segments  of  straight  lines 
connected  together.  The  coordinates  of  the  connecting  points  will  play  the  role  of  the 
interval  end  points  of  the  quantizers.  Let  us  first  write  an  expression  for  P^  and  P^.  in 
terms  of  these  point  coordinates.  If  this  expression  of  the  cost  is  minimized  with 
respect  to  each  coordinate  there  will  be  as  many  equations  as  the  number  of 
coordinates.  Solving  these  equations  simultaneously  yields  the  quantizer  parameters. 
This  is  the  core  of  the  QD  algorithm  which  is  summarized  as  follows: 
1.     Derive  the  threshold  equation  of  the  CD  system. 

A(yi,yO=C  (3.37) 


2.  Approximate  the  threshold  equation  by  a  stepwise  curve  satisfying  the  N  and  M 
constraints. 

3.  Write  an  expression  for  the  cost  in  terms  of  the  curve  parameters. 

4.  Minimize  the  average  cost  with  respect  to  the  curve  parameters. 

Let  us  illustrate  how  the  algorithm  works  for  the  case  of  detection  of  a  known 
signal  in  gaussian  noise. 

3.  An  Example:  The  Known  Signal  in  Gaussian  Noise 
Consider  Figure  3.2  when  y^  and  y2  are  given  by 

Hq  :  y.  =  nj 

Hj  :yj  =  aj  +  n.      ,i=l,2  (3.38)  , 
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Figure  3.2    Configuration  B,  The  Team  of  Two 
Quantizers  and  a  Fusion  Center. 
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CD  Curve 


QD  Approximation 


Figure  3.3     Quantized  Threshold  Curve. 
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where  the  a/s  are  positive  constants  and  N  =  [n^  nj  f  is  a  gaussian  random  vector  of 
zero  vector  mean  with  covariance  matrix: 


K=  ;  (3.39) 


■  [; :] 


It  is  required  to  design  the  N-level   and  the  M-level  quantizers    Q^  and  Qj 
and  the  decision  rule  y  where 

Y  =  YjqXY2q-*(0,l)  (3.40) 

to  minimize  the  average  decision  cost. 
Procedure  folloiving  the  QD  algorithm 

The  threshold  equation  of  the  CD  problem  has  been  shown  in  Chapter  II  to  have  the 
form: 

(a^  -p  a^  )y^  +(a2  +p  a^  )y^  =(aj2  +^^2  .p  ^^  ^^  )/2  +  (l-p-  )log(C).  (3.41) 


1.  The  CD  curve  is  a  straight  line  in  the  y^  y^  plane. 

2.  Possible  stepwise  approximations  for  the  threshold  equation  are  shown  in 
Figure  3.4  .  We  notice  that  in  Figure  3.4  a  and  c  the  two  quantizers  have  the 
same  number  of  quantizer  levels.  While  in  Figure  3.4  b  and  d  one  quantizer  has 
one  more  level  than  the  other.  From  Chapter  II,  we  can  expect  that  the 
constant  C  will  decide  the  superiority  of  a  or  c  and  of  b  or  d.  We  shall 
consider  optimum  parameters  of  Figure  3.4  a  and  b.  Similar  treatment  can  be 
considered  for  Figure  3.4  c  and  d.  In  Figure  3.4  a  the  point  Xj  =-oo  while  T^ 
is  finite.  In  Figure  3.4  b  X^  =  -co  and  T^  =  ^o  , 

3.  The  probabihty  of  detection  of  the  decision  rule  of  Figure  3.4  a  is  given  by 


i  i 

and  Pj.  is  given  by 
^'       X.  , ,    00 

Pf  =  Ii  l^;'''i^^yvy2i^o)<^yi<^y2'  (^.43) 

■^  i  i 
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For  the  detection  rule  of  Figure  3.4  b  P^  is  given  by  : 


(3.44) 


and  P^  is  given  by 


.X.,,,  .00 


^f^ilzk     ■'r.'^^i'^2/Ho)<Jyidy2 


(3.45) 


4.     Necessary  conditions  for  optimality  of  parameters  of  the  curve  in  Figure  3.4  a 
are: 


A(T.)=  C 


jJ^^yi/T,Ho)dy, 
_2j 

jJ^¥(y,/T.,H,)dy^ 


,i=l,2,...,N 


(3.46) 


and 


A(Xj)  =  C^ .i=2,3,...,N 


J^|-if(y,/X.,H,)dy, 


(3.47) 


For  Figure  3.4  b  ,the  optimality  conditions  are 


A  (Tj  )  =  C 


jJ^^y^/T.,Ho)dyi 
jJ^^y,/T.,H,)dy, 


.i=2,3,...,N 


(3.48) 


and 
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J/'fl:y2/Xi,H„)dyj 

A  (Xj )  =  C  — i ,i=  2,3 N  .  (3.49) 

J/'fl;y2/Xi,Hi)dyj 


The  last  two  equations  are  exactly  the  same  as  the  necessary  conditions  for 
optimizing  detection  using  a  Primary  Decision  Maker  and  its  quantized  second  opinion 
for  the  same  signals  in  gaussian  noise.  Recall  that  the  information  available  at  the 
PDM  is  more  complete  than  that  available  at  the  fusion  center  of  two  quantized 
observations.  Yet  the  two  problems  have  the  same  solution.  This  is  a  proof  of  the 
following  lemma. 
Lemma  3.2 

Optimum  detection  of  known  signal  in  gaussian  noise  using  two  quantized  observations 
of  N  and  N+1  levels  is  equivalent  to  optimum  detection  using  the  first  quantized 
observation  and  the  second  continuous  observation. 

Lemma  3.2  is  applicable  to  any  case  with  a  monotonic  likelihood  ratio.  This  can  be 
easily  proved  by  writing  the  necessary  conditions  of  optimality  for  the  two 
configurations.  A  special  case  of  Lemma  3.2  is  that  of  N  =  2.  It  corresponds  to  the 
tandem  configuration  of  two  detectors  in  a  Distributed  Detection  Network  (DDN)  [10]. 
The  "downstream"  detector  (decision  maker)  makes  its  decision  based  on  its  own 
observation  and  the  "upstream"  detector's  decision. 

D.      NUMERICAL  SOLUTION  FOR  THE  SYSTEM  PARAMETERS 

It    is    of  interest    to    compare    the    four    sets    of  equations    {(3.24    ),(3.26)}, 
{(3.32),(3.33)),  {(3.46  ),(3.47)}  and  {(3.48).(3.49)}  with  that  of  Lloyd  and  Max  [18,23]  for 
minimum  distortion  quantizer  parameters. 

Max's  trial  and  error  algorithm  to  solve  this  set  of  nonlinear  equations  can  be 
used.  However  Max's  algorithm  is  very  time  consuming  [24].  We  have  used  instead 
the  method  of  successive  substitutions  with  an  initial  guess  satisfying 

X2  ^  X3  <  ....^  Xj^  (3.50) 

and  put  the  equations  in  the  form 

Z  =  G(Z)  (3.51) 
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Figure  3.4    Possible  Approximations  of  tiie  Threshold  Equation. 
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The  kth  iteration  is  then  given  by 

Zk  =  G  (  Z^.l  )  (3.52) 

We  will  devote  the  next  chapter  to  solving  some  numerical  examples  using  this  method. 

E.       SUMMARY 

In  this  chapter  the  method  of  detection  using  quantized  sensor  observations  has 
been  introduced.  This  method,  referred  to  by  QD,  can  have  significant  performance 
improvement  compared  to  the  distributed  detection  algorithm  (DD)  with  only 
marginally  more  demand  on  the  communication  channels.  The  QD  algorithm  involves 
approximating  the  CD  threshold  hyperplane  by  a  stepwise  hyperplane  that  can  be 
spanned  with  the  quantized  data  and  that  minimizes  the  detection  cost. 

Also  the  equivalence  between  two  detection  configurations,  one  with  tandem 
connection  and  the  other  with  hierarchical  structure,  has  been  shown. 
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IV.  NUMERICAL  RESULTS 

In  this  Chapter  some  examples  are  solved  numerically  using  the  QD  algorithm. 
First  the  detection  of  known  signals  in  gaussian  noise  is  considered.  Next  detection  of 
signals  with  exponential  distribution  is  considered.  Finally,  the  algorithm  will  be 
applied  to  differentiating  between  gaussian  signals  with  different  variances. 

A.      KNOWN  SIGNAL  IN  GAUSSIAN  NOISE 

Again  consider  Figure  3.2  when  y^  and  y^  are  given  by 

Ho  :  Vj  =  nj 

Hi:yj  =  X  +  nj,i=l,2  (4.1) 

with  a^  =  4  and  ^2  =  2.  The  noise  vector 

N=[n^n2f  (4.2) 

is  of  zero  vector  mean  and  with  covariance  matrix  given  by: 

K  =       r^      ,n  (4-3) 


■  [; :] 


where  p  is  given  by 

p  =  E  {  n^  n^  ]  .  (4.4) 

It  is  required  to: 

1.  Design  the  primary  decision  maker  PDM  and  its  N-level  Quantizer  to  minimize 
the  average  decision  cost.  We  have  designated  this  structure  configuration  A. 

2.  Design  the  N-level  quantizers  Q.  and  Q2  and  the  decision  rule  u^  to  minimize 
the  average  decision  cost.   We  have  designated  this  structure  configuration  B. 

3.  Compare  the  performance  of  the  two  configurations  and  that  of  the  completely 
centralized  system. 

Following  the  algorithm  we  have: 

1.     The  threshold  equation  for  the  CD  problem  given  by, 
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(a^  -p  a^  )yi  +(a2  -p  a^  )y^  =(a/  +a/  -2p  a^  a2)/2  +  (l-p^  )log(C)  (4.5) 

a  straight  line  in  the  y^  y^  plane. 

2.  Figures  3.4  a  and  3.4  b  show  the  decision  regions  for  configuration  A  and 
configuration  B  respectively. 

3.  Probability  of  detection  P^j  and  probability  of  false  alarm  P^  of  PDM  are 
given  by: 

and 

Also  P^2  '^^"^  Pq  of  configuration  B  are  given  by: 


Pd2  = 


and 


:  ^   r^i^l      '    exp(-y,^'2)    erfc  f  ^  '  ';  '  "  1^'  '"'  ^  \  dv,        (4.8) 


Pc 


=iS(xr'7i"p^->'-''>  "H^^)}'"''     ''■'' 


4.     For   configuration   A   equations    of  the    quantizer   interval    end   points    and 
corresponding  PDM's  thresholds  for  the  gaussian  case  are  given  by 

„f  JVrP  ?^. -)  .  „  JT, -p  X, 
A(x,)  =  c       1^<'-P^)i \jiy-r)^ ,^^^3^^       ( 

\   v(i-p')     J     \  v(iy)      / 


and 
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IV(i-P^)    /       IV(iy)/ 

\     V(i-P^)         J        \    V(i-P^)  j 


A  (T  )  =  C V  ,^ ..  '    ^; r.       •''=  'A  N.     (411) 


For  configuration  B  the  quantizer  end  point  intervals  X's  and  T's  are  given  by: 


V(i-P^)/     i  VdW 


A  (Xj^  )  =  C X  \-:,, ,!:  \l^     -TTT— n       .^=  2,3,  N 

erfc 


and 


1   V(i-P-)  /        I  V(i-P-)J 

,..(■■  f^k- 1  -^■•P<TK-a,n      ,..f  JX,  -  a,  -  p  (T,  -a,  )  -, 

1   v(i-P^)     ;    \  vd-P^)      / 


erfc 
A(T,  )  =  C ^7-^-^-^—=^= .,"    ^   '"•/' ,k=2,3,  N 


We  have  solved  the  system  of  equations  of  the  two  configurations  using  the 
method  of  successive  substitution  for  N  =  2,  3,  4,  5  and  6  .  Figure  4.1  shows  the 
receiver  operating  characteristics  ROC  for  the  two  configurations  for  p  =  0,  for  different 
values  of  N.  The  ROC  for  the  CD  system  is  also  shown.  The  effect  of  p  is  illustrated 
in  Figure  4.2.  The  figure  shows  ROC  curves  for  Configuration  A  for  different  values  of 
N  and  for  p  =  0  and  0.25.  Figure  4.3  shows  the  average  cost  of  Configuration  B  and 
CD  vs.  C,  for  different  values  of  N.  The  relation  between  the  cost  of  detection  for 
Configuration  B  vs.  the  number  of  quantization  levels  is  shown  in  Figure  4.4  .  The 
figure  shows  the  exponential  decay  of  the  detection  cost  as  the  amount  of  information 
available  at  the  fusion  center  increases. 

The  following  results  are  noted  from  the  curves. 

1.  Configuration    A    has    better    ROC    curves    than    Configuration    B.     The 
performance  difference  is  large  for  N  =  2  but  gets  smaller  as  N  increases. 

2.  Both  performances  converge  to  that  of  the  CD  in  a  uniform  manner. 

3.  As  the  correlation  coefficient  increases  the  performance  difference  decreases. 
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Figure  4.1     ROC  Curves  for  Configuration  A  , 
Configuration  B  and  CD. 
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Figure  4.2     ROC  Curves  for  Configuration  A  for  p  =0  and  0.25. 
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Figure  4.3    Average  Detection  Cost  for  Configuration  B  and  CD. 
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Figure  4.4    Average  detection  cost  for  Configuration  B  vs.  N. 
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4.  As  N  increases  the  average  detection  cost  gets  smaller  and  tends  to  that  of  the 
CD.  Since  the  number  N  reflects  the  mutual  information  between  the  input  and 
the  output  of  the  quantizers,  the  relation  between  the  performance  degradation 
and  information  delivered  to  the  fusion  center  is  strong. 

B.       SIGNALS  WITH  EXPONENTIAL  DISTRIBUTIONS 

Consider  again  Figure  3.2  .   Let  y^  and  yj  have  the  following  distributions: 

.    Hq:  flyj)  =  5.0  exp(.  Xg  yj )  (4.14) 

and 

H^:  f(yj)  =  XjexpC-X^Vj  ).i=l,2  (4.15) 

and  assume  that  X^  is  less  than  Xq  .    It  is  required  to  design  the  quantizers  and  fusion 
rule  that  minimize  the  average  decision  cost. 
Following  the  QD  algorithm  we  have: 

1.  The  CD  threshold  equation  is  given  by 

Yl  +  y2  =  Cj  (4.16) 

where  C^  is  given  by 

C^  =    ^'^  C  .  (4.17) 

The  CD  threshold  equation  is  a  straight  line  in  the  first  quadrant. 

2.  Figure  4.5  shows  possible  approximations  of  the  threshold  equation.  For  N  =  2  , 
the  symmetry  suggests  equal  detector  thresholds.  For  N  ^  3  let  us  fix  X^  and 
T^-  to  zero. 

3.  The  probability  of  detection  and  probability  of  false  alarm  P^  and  P^.  are  given 
by 


Pd  =ij  [  exp(-X^  X.  )-  exp(-X^  X.^^  )  ]  Qxp{-X^  T  )  (4.18) 


and 
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IN 

V  =  I  [  exp(-Xo  Xj )-  exp(-Xo  Xj^ ^  )  ]  txpi-X^  Tj )  (4. 19) 

i=l 


4.  Writing  an  expression  of  the  average  cost  in  P^  and  P^  as  before  and  minimizing 
with  respect  to  Xj^  ,k=2,3,..N  and  Tj^  ,  k=l,2,..  N-1  one  obtains  the  set  of 
equations 

Xn         exp(-Xn  X.  )-exp(-Xn  X  ,  ,  ) 
exp{(Xn  -X,  )T.  }  =  -2-  C       ^     °     ^        "^     °     ^^        ,k=  1,2,...  N-1  (4.20) 

°     ^^        Xj  exp(-XiXj^).exp(-X^X^^l)  ' 

and 


Xn  exp(-Xn  T.  )-exp(-Xn  T.  ,  ,  ) 

exp{(Xo-X^)Xj^}=-^  C       ^\   °  .^'^      ^;   ',    !i^'\.k  =  2.3,...N.    (4.21) 
"      ^      ^        Xj  exp(-Xj  Tj^  )-exp(-  Xq  \+j  ) 

This  set  of  equations  Have  been  solved  by  the  method  of  successive 
substitutions  for  Xq  =  2,  X^  =1,  and  for  N=  2,3,4,5  and  6.  A  FORTRAN 
program  to  calculate  the  quantizer  parameters  is  given  in  Appendix  D. 

Figure  4.6  shows  ROC  curves  for  the  quantized  as  well  as  the  CD  systems.  The 
average  detection  cost  is  shown  in  Figure  4.7. 

We  note  the  following: 

1.  The  largest  performance  improvement  occurs  when  we  switch  from  N  =  2  to  N 
=  3  (  i.e.  only  less  than  one  more  information  bit  per  detection). 

2.  The  performance  curves  {  ROC(N)  }  and  (  R(N)  }  converge  uniformally  to  the 
performance  of  CD 

C.       GAUSSIAN  SIGNALS  WITH  DIFFERENT  VARIANCE 

Consider  again  the  structure  of  Figure  3.2  .   Let  sensor  observations  y^  and  y2  be 
independent,  identically  distributed  gaussian  random  variables  of  zero  mean.  However, 
under  Hq,  Var(yj)  =  (yQ^ 
and 

under  Hp  Var(yj  )  =  <t^^  ,i=  1,2.   For  specificity,  let 
<Tq  =  1  and  (T^  =  V  2  . 
Quantized  sensor  observations  are  sent  to  the  fusion  center  to  decide  which  of  the 
hypothesis  is  true.    It  is  required  to  design  the  quantizers  Q,  and  Q^  as  well  as  the 
fusion  rule  to  minimize  the  average  decision  cost. 
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Figure  4.5    Approximation  of  the  Threshold  equation  for 
DiiTerent  Values  of  N,  for  Exponential  Signals. 


84 


Figure  4.6    ROC  curves  for  Exponential  Signals. 
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Figure  4.7    Average  Detection  Cost  for  Different 
Values  of  N  for  Exponential  Signals. 
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Following  the  QD  algorithm  we  have: 

1.  The  CD  system  decision  rule  is  a  likelihood  ratio  test.  The  CD  detector  declares 
Hj  is  true  if 

Yl^  +72^  <  (l/2)log[<Ti2  /(ffg^  c)]/(aQ-2  .(y^-2  )-l  (422) 

otherwise  it  will  declare  Hq  is  true.  The  threshold  equation  is  the  circle 

where  Rg^  is  the  right  hand  side  of  inequality  (4.22  ). 

2.  Possible  approximations  of  the  CD  threshold  equation  are  shown  in  Figure  4.8  . 

3.  Figure  4.8  a  corresponds  to  3-level  quantizers.  The  corresponding  probability  of 
detection  and  probability  of  false  alarm  are  given  by; 

P^(3)=[erf(:-X/<Ti)p  (4.24) 

and 

Pf(3)  =  [erf(-X/(Jo)]2  (4.25) 

where  y^  and  ^^^  are  subdivided  by  the  points  X  and  -X.  For  the  5-level 
quantization  approximation  of  Figure  4.8  b,  the  probability  of  detection  and 
probability  of  false  alarm  are  given  by 

Pj  (5)  =  erfi:X3  l<5^  )  {2erf(:X2  l<5^  )-  zx^X^  /<7^  ) }  (4.26) 

and 

Pj.  (5)  =  erf(X3  /(Tg  )  {2erfIX2  /(Tq  )  -erf^Xj  /(Tg  )  }  (4.27) 

where  X2  ,  X^  ,-X.j  and  -X2  define  the  the  quantization  intervals  of  both  y^ 
and  y2  . 

4.  Inserting  P^.  (3)  and  P^  (3)  into  R  in  (3.3  )  and  minimizing  R  with  respect  to  X 
gives 


87 


(I,     erflX  /(Tf,  ) 

A  (X  )  =  C i-  —         Q  (4.28) 

(Tg    erfIX  /(Tj  ) 

Also  inserting  P^j(5)  and  P^5)  into  (3.3  )  and  minimizing  R  with  respect  to  Xj 
and  Xj  gives; 

(T,     erf(X,  /<Tn  ) 
A  (X^ )  =  C  -L     ;^^     Q     .  (4.29) 

^  ffg    erf(X3  /(Xj  ) 

<T,     erf(X,  /(In  )  -  erf(X,  /(!«  ) 

A  (X, )  =  C  — ^ Q^ ^     "^        .  (4.30) 

'    ^  '  Gq    erflX^  IG^  )  -  erf(X3  /c^  ) 

Solution  of  these  implicit  equations  in  the  quantizer  parameters  can  be  carried 
out  by  the  method  of  successive  substitution.  The  FORTRAN  program  to  calculate 
them  for  any  value  of  Cq  and  <J^  is  given  in  Appendix  F. 

Figure  4.9  shows  the  average  detection  cost  vs.  C  for  3-level  and  5-level 
quantizer  systems.  Detection  cost  of  CD  is  also  shown.  The  figure  shows  that  the 
detection  cost  decreases  dramatically  using  5-level  quantizers  m  comparison  to  3-level 
quantizers.  The  cost  of  the  CD  system  is  only  slightly  lower  than  that  of  the  5-level 
quantizers. 

Similar  procedures  can  be  carried  out  for  the  case  of  correlated  observations.  The  CD 
curve  in  this  case  is  an  ellipse  with  principle  axes  passing  through  the  origin.  It  can  be 
approximated  in  a  similar  way  as  the  circle. 

D.      CONCLUSION 

The  above  examples  show  the  uniform  convergence  of  the  Quantized  Detection 
Algorithm  to  the  Centralized  Detection  Algorithm.  The  Distributed  Detection 
Algorithm  is  a  special  case  of  QD.  It  follows  that  Quantized  Detection  is  an  efficient 
utilization  of  bandlimited  communication  channels. 
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Figure  4.8    Possible  Approximation  Of  the  Threshold  Equation. 
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Figure  4.9    Average  Detection  Cost. 
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V.  THE  CASE  OF  VECTOR  OBSERVATIONS 

A.  INTRODUCTION 

In  the  previous  two  chapters  the  QD  problem  for  the  case  of  scalar  sensor 
observations  was  solved.  It  is  now  time  to  extend  the  QD  algorithm  to  the  case  where 
each  local  observation  is  a  vector  Yj  .  The  QD  algorithm  can  be  applied  as  long  as  the 
corresponding  sufficient  statistic  for  the  centralized  detection  problem  can  be  divided 
into  local  statistics  to  be  quantized.  Let  us  consider  the  gaussian  case  and  put  it  in  the 
previous  framework. 

B.  QUANTIZED  DETECTION  WITH  VECTOR  OBSERVATIONS 

For  the  structure  of  Figure  5.1  the  observations  at  locations  1  and  2  are  given  by 


and 


iV. 


=  Aj  +  N.       ,i=  1,2. 


(5.1) 


Let  us  denote  the  observation  vector  by  Y 


y  = 


h 


li 


(5.2) 


The  noise  vector  N  ,  given  by 


N  = 


^l 


^2 


(5.3) 


is  multivariate  gaussian  with  zero  vector  mean  and  co variance 


R  = 


5i         ^12 


^21        ^2 


(5.4) 
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Figure  5.1     Vector  Observations. 
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R^  ,  ^2  ^^^  ^12  ^^^  ^^^  covariance  matrices  of  the  noises  at  locations  1  and  2  and  their 
common  covariance  matrix.  The  signal  vector  A  is  given  by 


Ai 


^2 


The  CD  system  decides  that  7  belongs  to  Z^  if  [6] 


(5.5) 


rl 


-1 


exp  {  (-1/2)  [  (7  -A)'^-'  (7  -A)  -7  'R-'  7  ]  }  ^  C 


(5.6) 


The  CD  threshold  equation  can  be  written  in  the  form 


A'^-l  7  =  log(C)  -(1/2)  A'^-^  A. 


(5.7) 


Using  the  block  matrix  inversion  lemma  [25],  (5.7  )  can  be  written  in  the  form 


a  II  +P  I2   =  log(C)-(l/2)A'5-^  A 


(5.8) 


In  (5.8  )  a  and  (J  are  given  by 


a 


~  Al  ^-1  '-12  -2     -21  ^ 

■A'2  (^2  '-21  -1      -21  ^     -21  -1 


(5.9) 


and 


P    =-A\{R^.R^2R^-^R,^y^R^^R^-^ 

+ A'2  {^2  -^21  ^1"'  ^12  y^ 


(5.10) 


Denoting 


li  =   a  Ii 


(5.11) 


and 


93 


^2=   P  X: 


(5.12) 


(5.8  )  becomes 


Ij  +  I2  =  log(C)-(l/2)A'5-^  A 


(5.13) 


where  1^  and  1^  are  bivariate  gaussian  with  zero  vector  mean  under  hypothesis  Hg 
Under  hypothesis  H,  their  vector  mean  is 


{" 


}■ 


a  Aj 


PA, 


(5.14) 


Covariance  of  1^  and  I2  is  given  by 


Cov(l^  ,l2  )  = 


a'  R^  a 


P'5,  P 


(5.15) 


In  (5.15  )  p  is  given  by 


=  a  ^12  P  V  ["(«  ^1  a  r^  (P  E2  P  )"M 


(5.16) 


The  distributed  signal  processing  is  to  form  local  linear  combinations  1^  and  1^  ,  then 
quantize  them  as  before.  This  processing  is  also  shown  in  Figure  5.1. 

C.      SUMMARY 

In  this  Chapter  it  is  shown  that  the  QD  algorithm  can  be  extended  to  the  case  of 
sensor  vector  observations.  An  application  is  the  case  of  high  quality  local  area 
communication  and  lower  quality  long  distance  communications.  In  this  case  sensor 
observations  in  local  areas  are  gathered  at  a  local  processor  to  form  the  local  sufficient 
statistics.  Quantized  local  statistics  are  then  sent  to  the  global  far  away  processor  for 
fusion. 
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VI.  OPTIMUM  ESTIMATION  USING  QUANTIZED  SENSOR 

OBSERVATIONS 


A.  INTRODUCTION 

In  the  previous  part  of  this  thesis  there  are  situations  in  which  a  group  of 
observers  make  local  decisions  that,  taken  in  combination  determine  the  overall 
performance  of  a  system.  The  observers  may  or  may  not  be  interconnected.  However, 
even  when  they  are,  for  a  variety  of  considerations  such  as  limitations  on 
communications  bandwidth,  transmitter  power,security,  or  perhaps  the  verv"  nature  of 
the  observers  themselves,  only  decisions  may  be  interchanged  between  them  and  not  all 
the  observations  upon  which  their  decisions  are  based  [1,5,26-33]. 

Another  case  of  interest  concerns  the  encoding  of  high  resolution  measurements 
for  transmission  between  observers  using  a  small  number  of  bits.  Here  a  remote 
observer  must  decide  which  of  N  possible  discrete  values  best  represents  his 
observation.  A  second  observer  is  to  combine  his  local  observations  with  the  discrete 
data  from  the  first  in  an  optimum  manner.  In  this  chapter  we  consider  the  problem  of 
regeneration  of  a  remote  sensor  observation  using  its  quantized  representation  and  a 
local  observation.  The  design  of  the  quantizer  at  the  remote  sensor  location  and  the 
optimum  linear  estimator  to  combine  the  quantized  data  with  the  local  observation  to 
minimize  the  expected  mean  square  estimation  error  will  be  considered.  Generalization 
of  the  results  to  the  vector  case  is  also  shown. 

B.  THE  LINEAR  MINIMUM  MEAN-SQUARE  ESTIMATE  OF  Y  ^ 

Consider  the  structure  of  Figure  6,1  in  which  the  observation  y^  is  quantized 
into  y,    by  a  quantization  rule  y 

7:yi-*yiq-  (6.1) 

The  quantized  data  y,    is  sent  to  sensor  S2  site. 

The  linear  minimum  mean  square  estimate  of  the  observation  y^  from  y^    and  '^^  is 

shown  in  (Appendix  F)  to  be 

y^   =      (1-  p2  )^  y^^  +   p    -A.  (i^  .p2  )y^      /(^  .J42  p2  ^  (^  2) 
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Figure  6.1     Estimation  Using  Quantized  Sensor  Observations. 
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where  we  have  assumed    that  y.   and  ^^  ^^^  random  variables  with  zero  mean  and 
variances 

E{yV  =  <y^i  ,i=l,2  (6.3) 

and  correlation 

P  =E(yiy2}/(Ti(T2  ■  (6.4) 

The  scalar  quantities  \\  and  ^  are  parameters  of  the  quantizer  and  are  given  by 


and 


i-'C.S^QkCkVV-  (6.6) 


N  is  the  number  of  quantization  levels  and  Cj^  and  Pj^  are  given  by 


Ck-d^'^S  f(yi )  dy,  )/Pk  (6-7) 


^v. 


and 


Pk=    l^^^^^O^-^l   '  (6.8) 


Qj^    is    the    k^   quantization   value   and    Qj^e[.Xj.,Xj^^j].     In   (6.7)    and    (6.8),    Xj^ 
,k=  1,2,. ...N,  are  the  quantization  interval  end  points,  with 

X^  =-co   and  Xj^^j  =  oo  . 
It  is  required  to  design  a  quantizer  that  minimizes  the  mean  square  estimation  error. 
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The  expected  mean  square  error  is  given  by  [34] 
where  7  isthe  vector 

I  =  [yiqy2f.  (6.10) 

Equation  (6.9  )  can  be  written  in  the  form  (Appendix  D) 

E(e2  }  =  (yj2  (i.p2  >,(i.^  ),(i.p2  (^  ^)  (6  11) 

where  co  is  given  by: 

(0=^-/n.  (6.12) 

A  plot  of  E{e2};(Tj"  vs.  (O  is  shown  in  Figure  6.2  for  p^  =  0,  0.25,  0.5,  0.75.  The  figure 
shows  that  the  mean  square  error  is  decreasing  with  co.  Recall  that  the  criterion  is  to 
minimize  the  mean  square  error. 

Equivalently  the  problem  now  is  to  maximize  (O  over  all  quantization  rules  where 


1    (|l,PkQ.c,)' 


CO 2 ^^ •  («-13) 


Appling  the  Cauchy  Inequality  [35]  to  the  numerator  yields 

( |L  ,Pr  Qk  Ck  )'  ^^(|,  \  <i^  \^,   P.  C,2  )  (6.14) 

with  equality  if  and  only  if 

Qk  =  q-  (6.15) 
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Figure  6.2     Relative  Mean  Square  Estimation  Error  vs.  (O. 
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Therefore 

(Tj    K^  1  ^     " 

gives  an  upper  bound  of  (o.  Equation  (6.15)  says  we  maximize  co  ,  and  thus  minimize 
E{e^}  by  making  the  quantization  level  Q^  equal  to  the  conditional  mean  of  y^  given 
that  y^  lies  in  the  kth  quantization  interval.  This  is  one  of  the  conditions 
characterizing  the  classical  Lloyd-Max  quantizer  [18,23.]  There  remains  the  problem  of 
how  to  pick  Xj^  ,k=l,2,...N,  so  that  the  upper  bound  of  (O  in  (6.16)  is  maximum. 
Notice  that  the  upper  bound  of  0  is  x\.  Therefore,  the  optimum  quantizer  will  be  a 
Lloyd-Max  quantizer  if  we  prove  that  maximizing  r\  over  all  choices  of  the  set  of 
points  (Xj^),  k=  1,2,...N,  is  equivalent  to  minimizing  the  distortion  E[{y^-y^  y}.   Since 

E{(yi-yiq)-}  =  c^^-2(j^\  +  <j^^r\  (6.17) 

=    (T^'(l-Tl) 

then  maximizing  r\  will  minimize  the  distortion  E((yj-y.  )^)  and  vice  versa.  Since  the 
Lloyd-Max  quantizer  is  the  optimum  quantizer  for  minimum  distortion  it  follows  that 
it  is  also  optimum  for  our  problem.  Accordingly  choose  Xj^  's  such  that  [23,18],  (see 
also  Appendix  G) 

V    =    ^k    _^^k-l  ,k=l,2,...N.  (6.18) 


Equations  (6.15)  and  (6.18)  along  with  (6.7)  completely  design  the  quantizer  [23,18]. 
Parameters  of  the  Lloyd-Max  quantizer  can  be  calculated  efficiently  using  the  method 
of  successive  substitution  (Appendix  G).  Values  of  E{e^  }/C7^^  vs.  N  are  listed  in  Table 
2  for  p  =0,  0.25,  0.5,  0.75.  The  table  shows  the  exponential  decay  of  the  MMSE  as  the 
number  of  quantization  levels  increases. 

Table  3  shows  a  comparison  between  the  average  number  of  bits  per  sample  used  in 
this  system  and  another  method  in  which  the  Maximum  Output  Entropy  (MOE) 
Quantizer  [36]  is  used.  Huffman  coding  [37]  is  assumed  for  both  quantizers. 
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TABLE  2 

MINIMUM  MEAN  SQUARE  ERROR  VS.  THE 
NUMBER  OF  QUANTIZATION  LEVELS 


N              p 

0 

0.25 

0.5 

0.75 

2 

0.3634 

0.3548 

0.3241 

0.2477 

4 

0.1175 

0.1166 

0.1131 

0.1021 

S 

0.0345 

0.0345 

0.0342 

0.0330 

16 

0.0095 

0.0095 

0.0095 

0.0094 

32 

0.0025 

0.0025 

0.0025 

0.0025 

64 

0.0006 

0.0006 

0.0006 

0.0006 

128 

0.00016 

0.00016 

0.00016 

0.00016 

TABLE  3 

COMPARISON  OF  THE  AVERAGE  NUMBER  OF  BITS  IN 
THE  MMSE  AND  THE  MOE  SYSTExMS 


N 

2 

4 

6 

8 

Optimum 
System 

I 

1.989 

2.4768 

2.8842 

MOE 

1 

2 

2.667 

3 
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C.  CONCLUSION 

The  trade  off  between  performance  and  communication  is  clear  from  Table  2. 
For  p  =0.5  the  relative  MMSE  is  0.75  without  communication.  This  corresponds  to 
substituting  co  =  0  in  (6.11  ).  The  relative  MMSE  decreases  to  0.32  using  one 
information  bit  per  sample.  The  relative  MMSE  is  0.11  using  two  bits/sample.  It  is 
only  0.03  using  3  information  bits/sample  (N=8)  and  is  0.00016  using  7  bits/sample. 
We  also  notice  that  for  high  number  of  quantization  levels  the  estimation  error  is 
approximately  equal  to  the  the  quantization  error.  This  means  that  the  estimator 
depends  mainly  on  y^  for  fme  quantization.  For  coarser  quantization  the  estimator 
depends  heavily  on  y2  to  reduce  the  MMSE.  Table  3  shows  that  the  designed  system 
has  considerable  reduction  in  the  number  of  bits  per  sample  compared  to  the  MOE 
quantizer  system. 

D.  GENERALIZATION  TO  THE  VECTOR  CASE 

In  this  section  we  will  consider  regeneration  of  a  random  vector  7j  from  its 
quantized  version  Y^  and  a  correlated  continuous  scalar  y2  .  As  an  application 
consider  a  sensor  S,  monitoring  the  activities  of  N  stations.  Due  to  some 
considerations,  perhaps  of  safety  nature,  only  simple  sensors  can  be  placed  near  the 
stations.  Because  of  other  considerations,  such  as  limited  bandwidth  communication 
channels,  only  quantized  sensor  measurements  can  be  sent  to  the  monitor.  Specific 
examples  can  be  the  case  of  monitoring  the  states  of  a  target  in  a  far  field  or  the 
positions  of  N  targets  in  a  multitracking  problem  [38,39].  Another  example  is  to 
monitor  the  radiation  levels  outside  of  N  nuclear  reactors.  A  third  example  is 
monitoring  the  activities  of  N  enemy  transmitters. 

Let  us  design  the  quantizers  at  the  N  sensor  sites  and  the  estimation  rule  at  the 
monitor  site  so  as  to  minimize  the  mean  square  error  of  each  component  of  J  .  Let  Y^ 
,the  sensor  observation  vector  be  given  by; 

£i=[yuyi2->-iNi'-  («'9) 

where  y^.  is  the  j^  sensor  observation  ,j=  1,2,. ..,N.  We  will  assume  that  components  of 
7  are  independent,  i.e. 

flYli  /Yij  ,72)=  ^^i  '%  )     'i  ^  i'  ^'J=  ^'2 >^-  (6-20) 

Under  the  above  conditions,  also  y^j  and  yj.    are  conditionally  independent  for  i  ^  j,  so 
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flYli /Ilq  .72  )=  flYii /Yiiq  .y2  )  .i=  1'2,...,N 


liq 


(6.21) 


The  MMS  estimate  of  7^  given  7.    is  given  by 


Ii=E{Zi/Zi,.y2} 


(6.22) 


Or 


h  = 


ifefe^:?dl 


LE(yiN  /^INq  '^2  L 


(6.23) 


Let  us  denote  the  error  vector  by  E  ,  so 


ff  =  [  e^  e^  ...e^  f 


(6.24) 


where  e.  is  the  error  in  estimating  y  ,i==  1,2,. ..,N.   The  MMS  error  covariance  matrix  is 


E{ir}  =E((7^-7|')(7j-7^)M. 


(6.25) 


The  trace  of  the  error  covariance  matrix  is  eiven  by 


trace(E(Fff^))  =Z    E^V  } 

1=  1 


(6.26) 


where 


e.  =v.  -Efv,.  /v..    ,y,  }. 
1       ^i        >^'  h  '  ^  hq  '■'2  •' 


(6.27) 


Minirhizing  the  trace  of  the  covariance  matrix  in  (6.26  )  is  accomplished  by  minimizing 
each  summand  alone  since  every  summand  is  nonnegative.  Now  assuming  Linear 
Minimum  Mean  Square  Estimation,  the  problem  of  minimizing  E  {e.^  }  implies  using 
the  Lloyd-Max  Quantizer  to  quantize  y^j  as  was  shown  previously. 
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In  conclusion  the  Linear  iMinimum  Mean  Square  Estimate  of  the  observation 
vector  y  implies  using  the  Lloyd-Max  quantizers  at  the  sensor  sites  and  the  same  linear 
combining  considered  in  the  scaler  case  at  the  central  processor. 
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VII.  SUMMARY,  RESULTS,  CONCLUSIONS  AND  SUGGESTIONS  FOR 

FUTURE  RESEARCHS 


A.       SUMMARY 

This  Thesis  begins  by  listing  some  reasons  why  Distributed  Signal  Processing  is 
more  practical  than  Centralized  Signal  Processing.  The  status  of  Distributed  Detection, 
an  important  case  of  Distributed  Signal  Processing,  and  its  complexity  are  reviewed  in 
Chapter  I. 

Chapter  II  deals  with  the  problem  of  optimum  fusion  of  local  decisions  into  a 
global  decision.  The  relationship  between  the  optimum  fusion  rule  and  the  ratio  of 
error  cost  is  shown.  The  dependance  of  the  performance  of  the  optimum  fusion  rule  on 
the  correlation  coefficients  between  sensor  observations  is  throughly  analyzed.  For 
higher  values  of  the  correlation  coefficients  the  Distributed  Detection  system  is  shown 
to  reduce  to  the  detector  of  the  highest  signal-to-noise  ratio. 

A  compromise  technique  between  Centralized  Detection  and  Distributed 
Detection,  Quantized  Detection,  is  suggested  in  Chapter  III.  The  main  issue  of  that 
chapter  is  to  control  the  degree  of  centralization  according  to  the  communications 
channel  constraints.  The  Quantized  Detection  technique  replaces  local  detectors  by 
quantizers  and  sets  a  global  fusion  rule  that  approximates  the  centralized  decision  rule. 
The  algorithm  matches  the  other  techniques  at  extreme  limits. 

Chapter  IV  contains  some  specific  applications  of  the  Quantized  Detection 
Algorithm  for  detection  problems.  A  significant  performance  improvement  is  attained 
by  replacing  Distributed  Detection  with  Quantized  Detection  with  three  quantization 
levels  (one  and  half  information  bits  per  sample  vice  one  information  bit  per  sample). 

Chapter  V  considers  applicability  of  the  Quantized  Detection  Algorithm  to  the 
case  of  vector  observations.  In  this  case  local  sufficient  statistics  are  quantized  in  the 
same  way  as  before. 

Chapter  VI  deals  with  the  regeneration  of  sensor  observations  from  their 
quantized  versions  and  another  correlated  observation.  The  local  quantizers  and  the 
optimum  linear  data  fusion  are  designed.  We  arrive  at  the  following  results  and 
conclusions. 
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B.       RESULTS 

1.  Detection  with  Distributed  Sensors 

a.  Optimum  Fusion  Rules  in  Distributed  Detection 

The  optimum  fusion  rule  depends  on  the  ratio  of  costs  of  different  types  of 
detection  errors.  For  high  cost  of  false  alarm  relative  to  the  cost  of  missing  the  target 
the  AND  fusion  rule  is  better  than  the  OR  fusion  rule,  and  vice  versa. 

The  performance  of  the  optimum  fusion  rule  depends  on  the  degree  of 
correlation  between  sensors.  The  performance  degrades  as  the  correlation  coefficient 
increases.  The  worst  performance  of  the  optimum  fusion  rule  is  at  and  above  a  critical 
value  of  the  correlation  coefficient  p^  .  In  that  region  of  correlation  the  best  system 
employs  only  the  detector  of  higher  signal-to-noise  ratio,  ignoring  the  lower  signal-to- 
noise  ratio  sensor  entirely.  The  performance  of  the  Distributed  Detection  system 
improves  as  the  signal-to-noise  ratio  imbalance  between  sensors  increases.  However 
there  is  a  large  performance  difference  between  the  Centralized  Detection  and  the 
Distributed  Detection  for  values  of  the  correlation  coefficient  above  p    . 

Below  p^^  the  performance  of  the  Distributed  Detection  system  improves  as 
the  correlation  coefficient  gets  smaller.  The  best  performance  (lowest  detection  cost)  of 
the  Distributed  Detection  system  is  achieved  at  p  =  -l.  Recall  that  the  Centralized 
Detection  system  has  perfect  detection  at  p  =-1.  This  is  due  to  the  efficient  use  of  the 
information  contained  in  two  observations  of  positive  signals  and  anticorrelated  noise 
samples. 

The  large  performance  difference  between  Centralized  Detection  and 
Distributed  Detection  systems  is  due  to  the  loss  of  information  in  the  local  detection 
processes.  As  a  remedy  to  the  performance  degradation  in  Distributed  Detection  we 
have  introduced  the  Quantized  Detection  Algorithm. 

b.  Quantized  Detection 

There  is  a  great  improvement  in  the  system  performance  using  Quantized 
Detection  with  three  quantization  levels  in  comparison  to  Distributed  Detection.  This 
performance  difference  between  Quantized  Detection  and  Distributed  Detection 
decreases  as  the  correlation  between  sensors  increases. 

The  Quantized  Detection  algorithm  is  applicable  to  the  case  of  vector 
observations  and  waveform  observations.  In  those  cases,  the  local  sufficient  statistics 
are  to  be  quantized  at  the  local  processor  and  transmitted  to  the  central  site  for  fusion. 
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The  Quantized  Detection  algorithm  is  implemented  by  the  quantizers  as 
local  processors  and  a  fusion  rule,  suggested  by  the  Centralized  Detection  decision  rule, 
at  the  central  site.  The  quantizers  used  in  the  Quantized  Detection  algorithm  are 
designed  to  minimize  the  detection  cost. 

2.  Minimum  Mean  Square  Estimation  in  Distributed  Sensor  Systems 

Minimum  mean  square  estimation  in  Distributed  Sensor  Systems  involves  the 
classical  Lloyd-Max  minimum  distortion  quantizers  at  the  local  levels  and  linear 
processing  at  the  global  central  level.  A  faster  iterative  algorithm  to  calculate  the 
Lloyd-Max  quantizer  parameters  is  the  method  of  successive  substitution.  It  also  has 
more  accurate  results  than  previously  reported  techniques. 

C.       CONCLUSIONS 

We  conclude  the  following: 

1.  Global  optimization  of  the  Distributed  Detection  implies  picking  the  fusion  rule 
and  corresponding  local  decisions  that  minimizes  the  detection  cost. 

2.  The  optimum  fusion  rule  in  Decentralized  Detection  depends  on  the  correlation 
coefficient  ,  the  a  priori  probabilities  and  the  ratio  of  costs. 

3.  For  optimum  fusion  of  two  local  unbalanced  decisions  there  is  a  particular 
value  of  p  that  decides  the  optimum  fusion  rule. 

4.  For  p  ^  p  OR  fusion  is  better  for  higher  cost  of  missing  target  while  AND 
fusion  is  better  for  higher  costs  of  false  alarm. 

5.  For  p  >  p^j.  the  optimum  fusion  rule  is  to  ignore  the  sensor  of  lower  signal-to- 
noise  ratio  and  optimize  the  decision  of  the  higher  signal-to-noise  ratio  sensor. 

6.  The  poor  performance  of  Distributed  Detection  compared  to  Centralized 
Detection  is  due  to  the  loss  of  information  at  the  local  levels. 

7.  The  Quantized  Detection  system  matches  the  Distributed  Detection  system  and 
the  Centralized  Detection  system  for  the  two  extreme  conditions  of 
quantization.  As  the  number  of  quantization  levels  increases  the  Quantized 
Detection  converges  to  Centralized  Detection. 

8.  The  Quantized  Detection  algorithm  has  a  tremendous  improvement  in 
performance  over  Distributed  Detection  even  with  only  3  quantization  levels. 

9.  The  performance  difference  between  Quantized  Detection  and  Distributed 
Detection  increases  as  the  correlation  of  the  observations  gets  smaller. 

10.  In  case  of  linear  Centralized  Detection  threshold  equations  in  the  observation 
space,  Distributed  Detection  and  Centralized  Detection  are  special  cases  of 
Quantized  Detection. 

11.  The  Quantized  Detection  algorithm  can  get  the  maximum  allowable 
performance  in  the  presence  of  communication  constraints. 
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12.  The  Quantized  Detection  algorithm  can  be  applied  to  arbitrary  distributions  for 
the  observations. 

13.  The  method  of  successive  substitution  is  applicable  to  the  design  of  many  types 
of  quantizers.  It  has  a  simple  programming  procedure  and  very  accurate  results. 

D.      SUGGESTIONS  FOR  FUTURE  RESEARCH 

The  following  are  some  areas  the  Quantized  Detection  algorithm  can  extend  to: 

1.  Optimum   detection   using   quantized   sensor    observations    for   the   case    of 
unknown  signal  in  noise. 

2.  Detection  of  M-ary  phenomena  using  quantized  sensor  data. 

3.  Utilizing  the  Quantized  Detection  algorithm  over  noisy  channels. 

4.  Illustration  of  the  relation  between  the  complexity  in  some  suitable  units  and 
the  amount  of  information  delivered  to  the  fusion  center. 

5.  Utilizing   the   Quantized  Detection   algorithm  to   meet  the  Neyman-Pearson 
criterion. 

6.  Extension  of  Distributed  Detection  and  Quantized  Detection  to  more  than  two 
sensors  with  correlated  observations  and  unequal  SNR's. 

7.  Development  of  general  principles  for  parsing  fusion  rules  given  a  Centralized 
Detection  surface  in  N-dimensional  space. 

8.  Application  of  the  Quantized  Detection  method  to  target  detection,classification 
and  tracking  using  distributed  sensors. 
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APPENDIX  A 

PROBABILITY  OF  DETECTION  AND  PROBABILITY  OF  FALSE 
ALARM  OF  THE  PRIMARY  DECISION  MAKER 

Given  that  the  Primary  Decision  Maker  (PDM)  receives  Q.  (the  j^  quantization 
level  of  the  Consultant  observation  y^),  and  that  its  own  observation  is  y2,  its 
observation  space  is  divided  into  two  decision  subspaces  Zy  and  Zg..  Let  us  denote  the 
conditional  probability  of  detection  and  probability  of  false  alarm  given  Q.  by  P^.  and 
P^  respectively.  P^.  and  P^  are  given  by: 

Pjj  =  Pr(Declare  H^  /y^^  =  Q.  M^  is  true)  (A.l) 

and 

Pf^  =  Pr(Declare  H^  /y^^  =  Q.  .Hg  is  true).  (A.2) 

These  can  be  expressed  as: 


di=|  ^z  «y, />•!,=  Qj,Hi)dyj  (A.3) 


and 


or  equivalently  as  , 

Pdi  =i  ^  z  J  ^  ffyi  .^2  ''>-i,  =  Qi  ."i  )''yi  dy2  (a-5) 

and 


^^   ^i    E  Z    ^      V^^l  '^2  /yiq  =Qi  'Ho  )dyi  ^72  (A.6) 
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But  n:yj  ,72  /Qj  .Hj )  is  given  by  [40] 

^     ^      J      '       (^0     otherwise 

where  Pr(Q.  /R  )  is  the  probability  of  the  j^j^  quantization  level  of  yj  under  hypothesis 
Hj .  It  is  given  by: 

Pr(Q./Hi)  =  J    ^^^^yi/Hj)  dy^  ,j=  1,2,...N  .1  =  0,1  (A.8) 

The  probability  of  detection  and  probability  of  false  alarm  are  now 

and 

Pf=S^j    Pr(Qj'Ho)P^  (A.  10) 

Inserting  (A. 5   and  (A. 6  )  into  (A.9  )  and  (A.  10  )  yields 


\X  i;;"i,ez,^^^'^-"^^'^^'^^  ^'-'^ 


and 


N        X.     ,     CO 

Pf  =  1  U*'i^  ,  pvyi  '"o )  dy,  Jyj  (3-5) 

M         '2         Ij 
where  Zj.  is  the  decision  region  Z^  given  that  y^  e  [  X.  ,X.^j  ]  . 


HO 


APPENDIX  B 

PROGRAM  LISTING  TO  CALCULATE  PARAMETERS  OF  THE  N  AND 

THE  (N  +  1)-LEVEL  QUANTIZERS 

In  this  appendix  we  give  a  program  listing  to  calculate  the  parameters  of  the  N 
and  the  (N+  l)-level  quantizers  in  configuration  A 

C   THIS  PROGRAM  CALCULATES  THE  OPTIMUM  N-LEVEL  AND  (N+1) -level 

C  QUANTIZER  PARAMETERS 

C  FOR  A  SYSTEM  OF  TWO  QUANTIZERS  AND  THEIR  FUSION  CENTER. 

C  THE  RATIO  OF  COSTS  C  RANGES  FROM  0.1  AND  10  . 

C  THE  VALUE  OF  N  =2,3,4,5  AND  6. 

C  THE  PROGRAM  USES  THE  MODIFIED  METHOD  OF  SUCCESSIVE  SUBSTITUTIONS  . 

REALMS  X(9),T(9),XX(9),TT(9),C,S1,S2,A,B,A12,A21,R,C1(20),R1(2) 
1 , AERR , RERR , ERROR , PD , PF , PDl ( 20 , 9 , 6 ) , PFl ( 20 , 9 , 6 ) , PD2 
1,PF2,PDC,PFC,X2,R3(20,2,6) ,X11 ,DCADER,F1 ,F2,APD, APF,A1 , A2 ,AP 

INTEGER  K,N,I,P,N1,N2,M,J,MX 

EXTERNAL  F1,F2 

COMMON  X11,R 

DATA  CI/ 10 . ODD , 9 . ODO , 8 . GDC , 7 . GDC, 6 . ODO , 5 . GDC, 4 . ODO, 3 . ODD , 2 . ODO , 
11.50DG, 
ll.GDG, .9GDG, .8GDG, .7GDG, .60D0, . 50D0, .40DG , .3GDG, .20D0 ,0 . lODO/ 

A1=1.GDG 

A2=2.GD0 

S1=1.GD0 

S2=1.0D0 

R1(1)=0.600D0 

R1(2)=0.S000D0 

AERR=G.ODO 

RERR=0.00010DG 

C     INITIAL  GUESS 

XX(1)=-9.50000DG 

XX(2)=-1.9000D0 

XX(3)=-1.50000DG 

XX(4)=-0.89000D0 

XX(5)=00.50000D0 

XX(6)=0.09000D0 

XX(7)=0G.99000D0 

XX(8)=01.50000DG 

XX(9)=2.09000DG 
C  TT(1)=4.67000D0 
C  TT(2)=3.89000D0 
C  TT(3)=2.67000D0 
C  TT(4)=2.29000DG 
C  TT(5)=1.90000D0 
C  TT(6)=1.49000D0 
C  TT(7)=1.G9000D0 
C  TT(8)=00.50000D0 
C     TT(9)=0.19000D0 

C   THE  FOLLOWING  INITIAL  VALUES  OF  T'S  CORRESPOND  TO  THE  CASE 
C   OF  CORRELATION  COEFFICIENT  GREATER  THAN  A1/A2 

TT(9)=4.67000D0 

TT(8)=3.89000D0 

TT(7)=2.67000D0 

TT(6)=2.29000D0 

TT(5)=1.90000D0 

TT(4)=1.49000D0 

TT(3)=1.G9000D0 

111 


TT(2)=00.50000D0 
TT(1)=0.19000D0 
DO  500  N=2,2 

WRITE(9,60)N 

DO  600   1=6,9 

C        INPUT  VALUE (S)  OF  THR  CORRELATION  COEFFICIENTS 
R=0.10D0'^DFLOAT(I) 
WRITE(9,61)R 
C        DO  200  M=l,20 
M=l 

C=C1(M) 
WRITE(9,61)C 

C        INPUT  MAXIMUM  NUMBER  OF  ITERATIONS  HERE 
MX=250 
K=0 
5  K=K+1 

IF  (K  .GT.  MX)  GO  TO  10 

IF  (K  .GT.l)  X(N)=XX(N; 

IF  (K  .GT.l)  T(N)=TT(N 

IF  (K  .GT.  1)X(N)=XX(N 

IF  (K  .GT.  1)T(N)=TT(N 

A=(S1*XX(2)-R'*^S2*TT(1))/(S1*S2*DSQRT((1.0D0-R**2)*2.0D0)) 


f/ 


A12=(S2'^A1-R*S1'*^A2)/(S1*S2'^DSQRT((1.0D0-R**2)*2.0D0)) 
A21=(S1*A2-R*S2*A1)/(S1*S2'*^DSQRT((1.0D0-R*'^2)'^2.0D0)) 

TT(l)=(A2*A2/2.0D0+DLOG(C)+DLOG((DERFC(A)- 
1  2.0D0)/(DERFC(A-A12)-2.0D0)))/A2 

T(1)=TT(1) 

A=(S1*TT(2)-R*S2*XX(2))/(S1*S2*DSQRT((1.0D0-R**2)*2.0D0)) 
B=(S2'^T(1)-R'^S1*XX(2))/(S2'*^S1'^DSQRT((1.0D0-R'^*2)^2.0D0)) 

XX(2)  =  (Al'^Al/2.0D0+DLOG(C)+DLOG((DERFC(A)- 
1      DERFC(B))/(DERFC(A-A21)-DERFC(B-A21))))/A1 

X(2)=XX(2) 

IF  (N  .EQ.  2)  GO  TO  17 

DO  15   P=2,N-1 
X(P)=XX(P) 

A=(S1'*^X(P)-R'^S2'^T(P))/(S1*S2*DSQRT((1.0D0-R**2)*2.0D0)) 
B=  S2^X(P+1)-R'^S1*T(P))/(S2*S1'^DSQRT((1.0D0-R^'^2)*2.0D0)) 

TT(P)  =  (A2'^A2/2.0D0+DLOG(C)+DLOG((DERFC(A)- 
1  DERFC(B))/(DERFC(A-A12)-DERFC(B-A12))))/A2 

T(P)=TT(P) 
A=(sm(P+l)-R'^S2*X(P+l))/(Sl'^S2*DSQRT((1.0D0-R**2)*2.0D0)) 
B=(S2^T(P)-R'^S1^X(P+1))/(S2*S1'^DSQRT((1.0D0-R**2)*2.0D0)) 

XX(P+l)=(Al*Al/2»0D0+DLOG(C)+DLOG((DERFC(A)- 
1  DERFC(B))/(DERFC(A-A21)-DERFC(B-A21))))/A1 

15     CONTINUE 
17    CONTINUE 

TT(N)  =  (A2'*^A2/2.0D0+DLOG(C)+DLOG(DERFC((XX(N)-R*TT(N))/ 
1DSQRT((1.0D0-R*^2) '^2.000)) 
1/DERFC((XX(N)-A1-R*(TT(N  -A2))/ 
1DSQRT(  (1.0D0-R'^*2)^2.0D0)) 
1))/A2 

C  CHECKING  THE  ACCURACY 
C  INPUT  REQUIRED  PRECISION  HERE 
AP=0.10d-07 

IF((DABS(T(N)-TT(N))  .GT.  AP) .OR. (DABS(X(N) -XX(N) )  .GT. 
1  AP))GO  TO  5 

10     CONTINUE 

X(1)=-10.0D0 
X(N+1)=10.0D0 
APD=0.0D0 
APF=0.0DO 
DO  81  Q=1,N 

A=X(Q) 

B=X(Q+1) 

X11=T(Q) 

APD=APD+0  .  50D0'^DCADRE  ( Fl ,  A ,  B  ,  AERR ,  RERR ,  ERROR ,  IREl ) 
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IF  (IREl  .NE.  0)  WRITE(6,60)IRE1 

APF=APF+0.50DO*DCADRE(F2,A,B,AERR,RERR,ERROR,IRE1) 
IF  (IREl  .NE.  0)  WRITE (6, 60) IREl 
81   CONTINUE 

PD1(M,I,N)=APD 

PF1(M,I,N)=APF 

R3(M,I,N)=1.0D0+C1(M)*APF-APD 

C  QUANTIZER  PARAMETERS 
DO  120  J=1,N 

WRITE(9,62)  X(J),T(J) 

120    CONTINUE 
C      WRITE(9,90) 

C200  CONTINUE 
C      WRITE (9, 90) 
C      WRITE(9,90) 

600  CONTINUE 
C      WRITE (9, 90) 
C      WRITE(9,90) 
C      WRITE(9,90) 

500  CONTINUE 

DO  501  J=l,2 
DO  502  1=1,20 

C*****A5<c*A***yc:ic**:*r*5t***^xx5>c*************A*5lt5«c***7<f  ************ 

C  OUTPUT  DETECTION  COST 

WRITE(10,61}C1(I),{  R3(I,J,N)  ,N=2,6) 

^***************  ******  ******:*'X**rc*7T7<:******x***  ******  :^*7r***** 

C  -  OUTPUT  PROBABILITY  OF  DETECTION  AND  PROBABILITY  OF 

C  FALSE  ALARM  FOR  DIFFERENT  VALUES  OF  N 

WRITE(8,61)(  PD1(I,J,N),PF1(I,J,N)  ,N=2,6) 
502      CONTINUE 

501  CONTINUE 

60  F0R1^T(1X,I4,4(1X,F15.8)) 

C90     FORMAT  (2X ,' CON  A  7^*'^'^**'^'^*****************************") 

61  FORMAT(1X',10(1X,F6.4)) 

62  F0RMAT(1X,2(1X,F15.8)) 

STOP 
END 

FUNCTION  F1(X) 

REAL*8  X,F1,A1,A2,R,X11,F11,F12 

COMMON  X11,R 

A1=1.0D0 

A2=2.0D0 

F11=DEXP(-(X-A1)**2/2.0D0)/ 
IDSQRT  ( 8  .  ODC^DATAN  ( 1 .  ODD ) ) 

F12=DERFC(  (X11-A2-(X-A1)'*^R)/ 
1(DSQRT(2.0DO'^(1.0DO-R**2)))) 

F1=F11^F12 

RETURN 

END 

FUNCTION  F2(X) 

REAL*8  X,F2,R,X11,F11,F12 

COMMON  X11,R 

F11=DEXP(-X'^*2/2.0D0)/ 
IDSQRT ( 8 . 0D0*DATAN ( 1 . ODO ) ) 

F12=DERFC((X11-X^R)/ 
IDSQRT ( 2. 0D0*(1.0D0-R**2))) 

F2=F11*F12 

RETURN 

END 
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APPENDIX  C 

PROGRAM  LISTING  TO  CALCULATE  PARAMETERS  OF  THE  TWO 

QUANTIZERS 

In  this  appendix  we  give  a  program  listing  to  calculate  the  parameters  of  the  two 
N-level  quantizers  in  Configuration  B. 

C   THIS  PROGRAM  CALCULATES  THE  OPTIMUM  N-LEVEL  QUANTIZER  PARAMETERS 

C  FOR  A  SYSTEM  OF  TWO  QUANTIZERS  AND  THEIR  FUSION  CENTER  . 

C  THE  CORRELATION  COEFFICIENT  IS  ASSUMED  TO  BE  LESS  THAN  A1/A2. 

C  FOR  THE  CORRELATION  COEFFICIENT  IS  GREATER  THAN  A1/A2  THE 

C  THE  PROGRAM  NEEDS  SLITE  MODIFICATIONS  ACCORDING  TO  THE  QD  ALGORITHM. 

C  THE  RATIO  OF  COSTS  C  RANGES  FROM  0.1  AND  10. 

C  THE  VALUE  OF  N  =2,3,4,5  AND  6. 

C  THE  PROGRAM  USES  THE  MODIFIED  METHOD  OF  SUCCESSIVE  SUBSTITUTIONS  . 

Q  *  *  *  ^ic  *  :*t  K  *  X  T^r  x  X  :*:  :1c  jic  X  X  7*1 7*:  X  7C  X  tIc  7*:  yc  *  X  A  :^  7*:  ■*  7*r  :*:  T^c  7C  A  :*:  jlc  tIc  7C  :^  :^  * 

REALMS  X(8),T(8),XX(8),TT(8),A1,A2,S1,S2,R,T12,T21,AA2,A,B 
1 , AERR , RERR , ERROR , PD , PF , PDl ( 20, 9 , 6 ) , PFl ( 20 , 9 , 6 ) , PD2 
1,PF2,PDC,PFC,C1(20),A21,A12,C,X2 
1 , XI 1 , DCADER , Fl , F2 , APD , APF , R3 ( 20 , 2 , 6 ) 

INTEGER  K,N,I,IER1,IER2,M,P,Q,L 

EXTERNAL  F1,F2 

DATA  CI / 10 . ODO , 9 . ODD , 8 . ODO , 7 . ODD , 6 . ODO , 5 . ODD , 4 . ODO , 3 . ODD , 2 . ODO , 
11.50D0, 
11. ODO, .90D0, .80D0, .70D0, .60D0, .50D0, .40D0, .30D0, .20D0 ,0 . lODO/ 

COMMON  Xll^R 

(;****x;^xxxxxxxx7^xx:<cyc7^:ic*7<c******yf*yc****3*:*:^5ic7<c**** 

C  INPIT  SIGNALS  HERE 

A1=4.0D0 

A2=2.0D0 
C  INPIT  VARIANCE  HERE 

S1=1.0D0 

S2=1.0D0 

C*7lcyc*7icA7lcX*X:i^XXX*>V7lc*5!c7lt:V****5V*5lt*5«tA**5^****5<r*********5<C*** 

C  INITIAL  GUESS  OF  THE  PARAMETERS 
XX(1)=-0.50000D0 
XX(2)=0.89000D0 
XX(3)=01.50000DO 
XX(4)=2.89000D0 
XX(5)=03.5GG00D0 
XX(6)=4.09000D0 

C   INITIAL  VALUES  OF  T'S  FOR  CORRELATION  COEFFICIENT  GREATER  THAN  A1/A2 
TT(1)=-0.67000D0 
TT(2)=0.89000D0 
TT(3)=01.67000D0 
TT(4)=2.89000D0 
TT(5)=03.50000D0 
TT  6)=4.89000D0 

C********xx:(5icyc:^r7icxx>k5^7it7icxx*:i^************************^ 

C   INITIAL  VALUES  OF  T ' S  FOR  CORRELATION  COEFFICIENT  LESS  THAN  A1/A2 
C   PUT  TT(1)  >  TT(2)> >TT(N) 

AERR=O.ODO 
RERR=0.00010D0 
DO  500  N=l,5 
DO  11  1=1,2 
R=DFL0AT(I-1)*0.250D0 
DO  20  M=l,20 
C=C1(M) 
K=0 
5    K=K+1 

114 


C  WRITE(6,60)K,T1,T11,T2,T22 

IF  (K  .GT.IOO)  GO  TO  10 
DO  25  L=1,N 
T(L)=TT(L 
X(L)=XX(L) 
25    CONTINUE 

XX(1)=(A1*A1/2.0DO+DLOG(C)+DLOG(DERFC((T(1)-R*X(1))/ 
1DSQRT((1.0D0-R*'^2)*2.0D0)) 
l/DgRFC  (T(l)-A2-Rf(X(l)-Al))/ 
1DSQRT((1.0D0-R**2)^2.0D0)) 
1))/A1 

IF  (N  .EQ.  1)  GO  TO  16 

A21=(S1'^A2-R^S2=^A1)/(S1*S2*DSQRT((1.0D0-R**2)*2.0D0)) 
A12=(S2*A1-R*S1*A2)/(S1*S2*DSQRT((1.0D0-R**2)*2.0D0)) 
DO  15  P=1,N-1 

A=(S1*X(P)-R*S2*T(P))/(S1*S2*DSQRT((1.0D0-R'^*2)*2.0D0)) 
B=(S2'^X(P+1)-R'^S1*T(P))/(S2'^S1*DSQRT((1.0D0-R'*^'^2)*2.0D0)) 
TT(P)=(A2*A2/2.0D0+DLOG(C)+DLOG((DERFC(A)- 
1  DERFC(B))/{DERFC(A-A12)-DERFC(B-A12))))/A2 

A=  (S1*T (P+1 ) -R*S2'^X(P+1 ) )  /  ( S1*S2'^DSQRT ( ( 1 .  0DO-R**2 ) *2 . ODD ) ) 
B=(S2'^T(P)-R'^S1*X(P+1))/(S2*S1*DSQRT((1.0D0-R**2)*2.0D0)) 
XX(P+1)=(A1*A1/2.0DO+DLOG(C)+DLOG((DERFC(A)- 
1  DERFC(B))/(DERFC(A-A21)-DERFC(B-A21))))/A1 

15  CONTINUE 

16  CONTINUE 

TT(N)=(A2*A2/2.0DO+DLOG(C)+DLOG(DERFC((X(N)-R*T(N))/ 
IDSQRT  ( ( 1 .  ODO-R'^'^^2  )  ^2  .  ODO ) ) 
1/DERFC((X(N)-A1-R^(T(N)-A2))/ 
1DSQRT((1.0D0-R**2) '^2.000)) 
1))/A2 

IF((DABS(T(N)-TT(N))  .GT.  0.10D-05).OR.(DABS(X(N)-XX(N))  .GT. 
1  0.10D-05))GO  TO  5 

10  CONTINUE 
X(N+1)=10.0D0 
APD=O.ODO 
APF=O.ODO 

DO  81  Q=1,N 
A=X(Q) 
B=X(Q+1 
X11=T(0 

APD=APD+0 . 50D0*DCADRE ( F1 , a , B , AERR , RERR , ERROR , IREl ) 
IF  (IREl  .NE.  0)  WRITE(6,60)IRE1 

APF=APF+0  .  SODC'^DCADRE  (  F2  ,  A ,  B  ,  AERR ,  RERR ,  ERROR ,  IRE  1 ) 
IF  (IREl  .NE.  0)  WRITE(6,60)IRE1 
81   CONTINUE 

PD1(M,I,N)=APD 

PF1(M,I,N)=APF 
R3 (M, I ,N)=1 . ODO+Cl (M)*PF1 (M, I ,N) -PDl (M, I ,N) 

C   OUTPUT  SYSTEM  PARAMETERS 
DO  120  J=1,N 

WRITE(9,62)  X(J),T(J) 
120   CONTINUE 
C         WRITE(9,61)C,T(1),T(2),APD,APF 
C         WRITE(8,61)APD,APF 

C         WRITE(6,61)C,X(1),X(2),APD,APF,PDC,PFC 
20     CONTINUE 

11  CONTINUE 
C      WRITE(9,90) 

WRITE (8, 90) 
500   CONTINUE 

C  OUTPUT  AVERAGE  COST 

DO  501  J=l,2 

DO  502  1=1,20 

WRITE(10,61)C1(I),(  R3(I,J,N)  ,N=1,5) 

C  OUTPUT  PROB.  OF  DETECTION  AND  PROB.  OF  FALSE  ALARM 
WRITE(8,61)(  PD1(I,J,N),PF1(I,J,N)  ,N=1,5) 
502   CONTINUE 
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WRITE(8,90) 
501   CONTINUE 

60  F0RMAT(1X,I2,9(1X,F6.4)) 

90  F0RI-IAT(2X    '  CON  B      ****''?'<******************************' 

61  FORMAT(lx!lO(lX,F6.4)) 

62  F0R14AT(1X,2(1X,F15.8)) 

STOP 
END 

FUNCTION  F1(X) 

REALMS  X,F1,A1,A2,R,X11,F11,F12 

COMMON  X11,R 

A1=4.0DG 

A2=2.0D0 

F11=DEXP(-(X-A1)**2/2.0D0)/ 
IDSORT  ( 3  .  ODC^DATAN  ( 1 .  ODO )  ) 

F12=DERFC({X11-A2-(X-A1)*R)/ 
1(DSQRT(2.0D0'^(1.0D0-R**2)))) 

F1=F11*F12 
•  RETURN 

END 

FUNCTION  F2(X) 

REAL'^S  X,F2,R,X11,F11,F12 

COMMON  X11,R 

F11=DEXP ( -X**2/2 . ODO) / 
IDSQRT ( 8 . ODO^DATAN ( 1 . ODO ) ) 

F12=DERFC((X11-X'^R)/ 
1DSQRT(2.0D0^{1 .0D0-R'^*2) ) ) 

F2=FII^F12 

RETURN 

END 
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APPENDIX  D 

PROGRAM  LISTING  TO  CALCULATE  PARAMETERS  OF  THE  TWO 
QUANTIZERS  FOR  THE  CASE  OF  EXPONENTIAL  DISTRIBUTIONS 

c  ************************* 

C   THIS  PROGRAM  CALCULATES  THE  OPTIMUM  N-LEVEL  QUANTIZERS  OF  TWO  SEN- 

C  SOR  OBSERVATIONS  OF  EXPONENTIAL  DISTRIBUTIONS 

C  TO  MINIMIZE  A  GLOBAL  SYSTEM  RISK  FOR   FUSION 

C  SEE  CHAPTER  IV 

Q  ************************* 

REALMS  X(3),T(8),XX(8),TT(8),PD,PF,PD1(20,9,6),PF1(20,9,6),PD2 
1 , PF2 , PDC , PFC , CI ( 20 ) , C , X2 , Xll , APD , APF , R3 ( 20 , 2 , 6 ) , AL , BL , Y 
1,APC(20,2),AFC(20,2),R4(20,2),AD2,AF2,RR2,AP 

INTEGER  K,N,I,IER1,IER2,M,P,Q,L,MX 

DATA  CI/ 50 . ODO , 40 . ODO , 30. ODO, 20. ODO , 15 . GDC, 10. ODD , 9 . ODO , 
18 . ODO , 7 . ODO , 6 . ODO , 5 . ODO , 4 . ODO , 3 . ODO , 2 . GDO , 1 . 50D0 , 
11. ODO, .90D0, .80D0, .7QD0, .60D0/ 
C  ************************* 

C  INITIAL  VALUES  OF  QUANTIZER  PARAMETERS 
C  FIRST  QUANTIZER 

XX(1)=00.00000D0 
XX(2)=0.89000D0 
XX(3)=01.50000D0 
XX(4)=1.89000D0 
XX(5)=02.50000D0 
XX(6)=3.09000DG 
C  SECOND   QUANTIZER 

TT(1)=3.G90GG0D0 
TT(2)=2.50000D0 
TT(3)=01.890G0DG 
TT(4)=1.50000DG 
TT(5)=G0.89000D0 
TT(6)=0.0G00GD0 
DO  5GG  N=3,6 
DO  11  1=1,2 
C  PARAMETERS  OF  THE  EXP.  DISTRIBUTIONS 
AL=DFL0AT(I)*0.5GD0 
DO  20  M=1,2G 
C=C1(M) 
DO  55  P=1,N 

TT(P)=DLOG(2.0D0*C)*DFLOAT(N-P+l)/DFLOAT(N)/AL 

XX(N-P+l)=DLOG(2.0D0'^C)*DFLOAT(N-P+l)/DFLOAT(N)/AL 
55   CONTINUE 

K=0 
5    K=K+1 

DO  89  PP=1,N 

T(PP)=TT(PP; 

X(PP)=XX(PP 

89    CONTINUE 
C********************* 

C    INPUT  MAXIMUM  NUMBER  OF  ITERATIONS 

MX=200 

IF  (K  .GT.MX)  GO  TO  10 
C     XX(1)=DLOG(2.0DO*C)/AL-TT(1) 

XX(1)=0.GDG 

TT(l)  =  (DLOG(DEXP(-AL'^XX(l))+DEXP(-AL*XX(2)))+DLOG(2.GD0'^C))/AL 
DO  15   P=2,N-1 

XX(P)  =  (DLOG(DEXP(-AL*TT(P))+DEXP(-AL'^TT(P-1)))+DLOG(2.GDO*C))/AL 

TT(P)  =  (DLOG(DEXP(-AL*XX(P))+DEXP(-AL'^XX(P+l)))+DLOG(2.0D0*C))/AL 
15  CONTINUE 

XX(N)  =  (DLOG(2.GDO'^C)+DLOG(DEXP(TT(N))+DEXP(-AL*TT(N-1))))/AL 
C     TT(N)=DLOG(2.GDO*C)/AL-XX(N)    ^   ^  ^^      ^ 

TT(N)=0.0D0 
C  ACCURACY  CHECKING 
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C  INPUT  PRECESSION  HERE 
AP=0.10d-05 

IF((DABS(XX(N)-X(N))  .GT.  AP) .OR. (DABS(TT(l)-T(l) )  .GT. 
1  AP))GO  TO  5 

10  CONTINUE 
WRITE(8,60)  K 

BL=2,0D0'^AL 
APD=O.ODO 
APF=O.ODO 
DO  81  Q=1,N-1 
APD=APD+DEXP(-AL*TT(Q))*(DEXP(-AL*XX(Q))-DEXP(-AL*XX(Q+1))) 
APF=APF+DEXP(-BL*TT(Q))*(DEXP(-BL*XX(Q))-DEXP(-BL*XX(Q+1))) 
81   CONTINUE 

APD=APD+DEXP(-AL'^X(N)) 
APF=APF+DEXP(-BL*X(N)) 

APC(M,I)=(1.0D0+DLOG(4.0D0*C))/(4.0D0*C) 
AFC (M , I )= ( 1 • 0D0+DLOG( (4 . ODO*C) *^2) )/ (4 . 0D0*C)**2 
PD1(M,I,N)=APD 
PF1(M,I,N)=APF 
R3(M,I,N)=1.0D0+C1(M)*PF1(M,I,N)-PD1(M,I,N) 
R4(N,I)=1.0D0+C1(M)^AFC(M,I)-APC(M,I) 
DO  120  J=1,N 

C  OUTPUT  QUAInITIZER  PARAMETERS 

WRITE(9,62)  XX(J),TT(J) 
120    CONTINUE 

WRITE(9,90) 
20     CONTINUE 

11  CONTINUE 
WRITE (9, 90) 

C      WRITE (9, 90) 
WRITE(8,90) 

500  CONTINUE 

DO  501  J=l,2 
DO  502  1=1,20 
AP2=1.0D0/(2.0D0*C1(I))  . 
AF2=AP2^'^2 
RR2=1.0D0+C1(I)*AF2-AP2 

C     OUTPUT  DETECTION  COST 

WRITE(10,61)C1(I) ,RR2,(  R3(I,J,N)  ,N=3 , 6 ) ,R4(I , J) 

C      OUTPUT  PROS.  OF  DETECTION  AND  PROB .  OF  FALSE  ALARM 

WRITE(8,61)AP2,AF2,(  PDl (I , J ,N) ,PF1 (I , J,N)  ,N=3 , 5) ,APC(I , J) 
1,AFC(I,J) 
502   CONTINUE 

WRITE(8,90) 

501  CONTINUE 

60  F0RIli^T(lX,I6,9(lX,F6.4)) 

90      FORMAT  (2X  'CON  B   ■^■^T^*:7^Tr-f^-f^-f^*:-^-k-k**-k-k-k-k-k-ki^-k-k-k-k-k-k-k-k:k:k-kik-k-k>  \ 

61  FORMAT  lx!lO(lX,F6. 4; ' 

62  F0RMAT(1X,3(1X,F15.8; 

STOP 
END 
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APPENDIX  E 

PROGRAM  LISTING  TO  CALCULATE  PARAMETERS  OF  THE  TWO 

QUANTIZERS  FOR  EXAMPLE  3,  CHAPTER  IV 

I 

C   THIS  PROGRAM  CALCULATES  THE  OPTIMUM  QUANTIZER  PARAMETERS  OF  TWO 

C  N-LEVEL  QUANTIZERS  IN  ORDER  TO  MINIMIZE  A  GLOBAL  SYSTEM  RISK  FOR 

C  DETECTION  OF  SIGNALS  WITH  DIFFERENT  VARIANCE.  N=3  ,  N=5. 

C   SI  =  SIGNAL  VARIANCE  UNDER  HI 

C   SO  =  SIGNAL  VARIANCE  UNDER  HO 

C   Tl  =  QUANTIZATION  POINT  FOR  N=3  (  T1,-T1  ) 

C  XI, X2  QUANTIZATION  POINTS  FOR  N=5  (XI ,K2 , -XI , -X2) 

C   C|(20)  ARRAY  OF  RATIO  OF  COSTS 

c  K  =  Number  of  iterations. 

REAL*8  T1,T11,S1,S2,R,C,X2,PD,PF,C1(15),Z31,Z21,X3,X33,X22 
1,TS,SSS,TTT(10),X1,PD3,PF3,PDC,PFC,R2,R3,RC 
INTEGER  K,N,K1 

DATA  Cl/Ol.ODO, .900D0, .800D0, .700D0 ,0 .60D0 ,0. 50D0 ,0.40D0 , 
10. 30D0, 0.20D0, 0. lODO, .090D0, .08GD0, .07000,0. 060D0,0.050D0/ 
S1=1.0D0 
S0=DSQRT(2.0D0) 
C  INITIAL  VALUES  OF  THE  QUANTIZER  PARAMETERS 
T1=01.15800D0 
X1=01.15800D0 
X2=00.15800D0 
WRITE(6,60)K,S1,S0,C 
WRITE(9,60)K,S1,S0,C 
WRITE(8,60)K,S1,S0,C 
SSS=(1.0D0/S0'^'^2-1.0D0/S1**2)/2.0D0 
DO  101  N=2,2 
DO  100  1=1,15 
C=C1(I) 
K=0 
5    K=K+1 

IF  (K  .GT.IOO)  GO  TO  10 
T11=T1 

TS=(DL0G(S1/S0)+ 
1DLOG(C)+DLOG{(DERF(T1/DSORT(2.0DO) 

1/SO))/(DERF(T1/DSQRT(2.0DO)/S1)))*DFLOAT(N-1))/SSS 
C     IF  (TS  .GT.  O.ODO)  T1=DSQRT(TS) 
T1=DSQRT(DSQRT(TS)) 

IF((DABS(T1-T11)  .GT.  0.10D-05))  GO  TO  5 
CONTINUE 
10     TTT(N)=T11 
K1=0 
55    K1=K1+1 

IF  (Kl  .GT.IOO)  GO  TO  15 
a33=X3 
X22=y2 

X2=(DL0G(S1/S0)+ 
1DLOG(C)+DLOG((DERF(X3/DSQRT(2.0DO) 
1/S0))/(DERF(X3/DSQRT(2.0D0)/S1))))/SSS 
C     IF  (TS  .GT.  O.ODO)  T1=DSQRT(TS) 
X2=DS0RT(X2) 
Z30=DERF(X3/DSQRT(2.0D0)/S0' 
Z20=DERF(X2/DSQRT(2.0D0)/S0' 
Z31=DERF(X3/DSQRT(2.0D0)/S1' 
Z21=DERF(X2/DSQRT(2.0D0)/S1' 
X3=(DLOG(S1/S0)+ 
1DLOG(C)+DLOG((Z30-Z20)/(Z31-Z21)))/SSS 
X3=DSQRT(DABS(X3)) 

IF(((DABS(X2-X22   .GT.  O.lOD-05))  .0R.((  DABS(X3-X33)  .GT, 
10.10D-05)))  GO  TO  55 
CONTINUE 
15     TTT(N)=T11 
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TS=-DLOG(SO**2/S1**2/C)/SSS 

PDC=1,0D0-DEXP(-TS/(2.0D0'^S1*S1)) 

PFC=1.0D0-DEXP(-TS/(2.0D0*S0*S0)) 

RC=1.0D0+C*PFC-PDC 

WRITE(9,60)K,T1,T11,X2,X22,X3,X33 

WRITE(6,60)K,T1,T11 
PD=DERF (Tll/DSQRT ( 2 . ODO ) /SI ) **N 
PF=DERF(T11/DSQRT(2.0D0)/S0)**N 

R2=1.0D0+C'^PF-PD 
PD3=(DERF(X22/DSQRT(2.0D0)/S1)*2.0D0-DERF(X33/DSQRT(2.0D0)/S1)) 
1  *DERF(X33/DSORT(2.0DO)/S1) 

PF3=(DERF(X"'22/DSQRT(2.0D0)/S0)*2.0D0-DERF(X33/DSQRT(2.0D0)/S0)) 
1  *DERF(X33/DSQRT(2.0D0)/S0) 

R3=1.0D0+C^PF3-PD3 

WRITE(8,60)N,PDC,PFC,PD,PF,PD3,PF3 

WRITE(10,60)N,C,R2,R3,RC 
;  IF  ((I  .EQ.  1)  .OR.  (l  .EQ.  10  ))  WRITE(10,60)N,C,R2 ,R3 ,RC 

WRITE(6,60)N,C,TTT(N),PD,PF,PD3,PF3 

100  CONTINUE 

101  CONTINUE 

60    FORMAT(1X,I3,6(1X,F10.7)) 
STOP 
END 
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APPENDIX  F 
LINEAR  MINIMUM  MEAN  SQUARE  ESTIMATE  OF  Y^ 

Having  7  =[  y,     y2  f  the  LMMS  estimate  of  y,  and  the  corresponding  mean 
square  error  are  given  by  [34]: 


Yi  =  E{yi  r  mi  Y'  ]■'  I 


(F.l) 


and 


E{e2  }  =  Ely^^  ^.^{y^  yt  ]E(y  t  Y^  E(7  y^  ] 


(F.2) 


where 


E{yir}  =  [E(yiy,    }     E(y^  y^  }  ] 


(F.3) 


and 


E{lt]  = 


E(y2yiJ  E( 


:y^2)  J 


(F.4) 


The  entries  of  these  matrices  are: 


E(yiy.<,)=jS'i'iQiE{yi''yiq=Q,) 

N 

=y  p.  Q.  c. 


(F.5} 


Ni 


E(y2yiq)=lPjQjE(y2/yiq=Qj) 


j  =  i 


(F.6) 


but 
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E(y2  /yiq  =  Qj ) = E{{y2/  71  }/yiq  =  Qj )  (f-7) 

For  the  case  where  y^  and  ^^  are  jointly  gaussian,  we  can  write 

E(y2yiq}=p<^i^2SPjQjS/^i'  (F-8) 

^^y2yiQ^=p^i  *^2^*  (^•^) 


2  1    _• 


E^V^72nPiQi2  (p-iO) 


=  '^1' n 


Inserting  these  in  (F.l  )  and  (F.2  )  and  performing  matrix  multiplications  yields 

vj  =  1(1-  p2  )n  y^^  +  p  -^  (n  -p^  )y2l  /(n  -fi^  p^ )  (6.2) 

and 

E(e2)  =  (yf  (ly  )(l-co)/(l-p2co)  (6.11) 

where  (O  is  given  by: 

co=n2/Ti.  (6.12) 
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APPENDIX  G 

SOLUTION  OF  THE  LLOYD-MAX  QUANTIZER  PARAMETERS  BY 
THE  METHOD  OF  SUCCESSIVE  SUBSTITUTION 

1.  INTRODUCTION 

The  minimum  distortion  quantizer  parameters  [18,23],  as  well  as  parameters 

based  on  other  criterion  such  as  quantizers  for  signal  detection  [41],  minimum  risk 

"quantizers  and  quantizers  for  LMMS  estimation  error  dealt  with  in  this  thesis,  can  be 

solved    by    Max's    trial    and    error    technique    [18].     There    are    also    many    other 

approximation  methods  to  calculate  the  quantizer  parameters  [42],  [43]  and  [44]. 

In  this  Appendix  we  apply  the  method  of  successive  substitution  and  its 
modifications  [19]  to  solve  for  the  Lloyd- Max  quantizer  parameters.  It  is  more  accurate 
and  computationally  more  efficient  than  the  previously  reported  methods.  It  is  shown 
to  easily  generate  7  bit  (128  level)  optimum  quantization. 

2.  STATEMENT  OF  THE  PROBLEM 

The  Lloyd- Max  minimum  mean  square  distortion  quantizer  problem  deals  with 
transforming  a  random  variable  X  of  difierentiable  probability  density  function  f(x) 
into  the  N-level  discrete  random  variable  Y. 

Y(X)  =  Y.forXe[Xj,Xj^J  (G.l) 

The  optimum  parameters  minimize,  the  distortion  D 

D  =  ^r''"U-yj)2fi:x)dx  {G.2) 

1=^    X.  ' 

1 

with 

.00    =X       ^    X       <  <    X        <    X  =00 

Differentiating  D  with  respect  to  x.  and  y.  yields  the  following  necessary  conditions  of 
optimality  : 

^i=(yi  +yi  +  l)/2   ,i=2,3,...N  (G.3) 
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x,+i ,  x= 


y.  =(J   •+lxirx)dx)/(f    '+1   f(x)dx),i=l,2,..N  (G.4) 

a  set  of  simultaneous  equations  of  propagating  character.  That  is,  if  y^  is  chosen 
correctly  then  X2  can  be  calculated  from  (G.4  ),  y2  from  (G.3  ),  x^  from  (G.4  )  and  so 
forth  [18].  In  this  case  the  value  of  yj^  calculated  from  (G.3  )  must  agree  with  its  value 
calculated  from  (G.4  )  with  Xj^^^  =00  .  This  was  the  core  of  Max's  trial  and  error 
algorithm;  to  pick,  a  value  for  y^  and  calculate  the  parameters  up  to  and  including  y,^  , 
which  must  agree  with  the  value  of  y>^  calculated  from  (G.4  ),  otherwise,  to  pick 
another  value  of  y^.   Let  us  put  the  system  of  equations  in  the  form 

Z=G{Z)  (GJ) 

where  Z  is  a  2N-1  vector  given  by: 

z=[yiX2y2 y^r]'  (G.6) 

and  apply  the  iterative  substitution 

Z        =G{Z  ,.)  (G.7) 

—new       —  ^— old  ^  ^        ^ 

with  a  suitable  initial  guess.  The  convergence  is  guaranteed  if  d  G^  Id  Z.  is  sufficiently 
small  for  ever\-  k,j  =  1,2 2N-1  [19].    From  (G.4  ) 

d  G.  !d  y.  =  [{X.  ^  ^  -y.  )[{x.  +  ^  )  +  (y^  -x.  )f(x.  )V{2?.  )  (G.8) 

where  P.  is  the  probability  the  input  of  the  quantizer  is  in  the  j^^  interval. 

P.  =fi  +  l    n;x)dx.  (G.9) 

J       x. 
J 

The  numerator  in  (G.8  )  is  an  approximation  of  the  integral  in  (G.9  )  by  the 
trapezoidal  rule  with  the  subdivision  [x.  ,y.  ,  x.  ^  ^  ],  so  the  value  of  the  derivative  is 
ver>'  likely  less  than  one.  Also,  substituting  for  y  and  y.^^  in  (G.3  )  from  (G.4  )  and 
differentiating  with  respect  to  x.  it  is  easily  to  show  that 
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d  a  /a  X.  =(y.  -Xj  )f^Xj  )/(2Pp  +  (Xj  -Yj.i  )fi:x.  )/2P..j  (G.IO) 

which  is  less  than  {d  Q.  jd  y.).  The  method  can  be  more  efficient  if  we  use  the  updated 
values  in  the  same  iteration.  In  this  modification  of  the  method  the  best  current 
values  of  the  parameters  are  used.  This  choice  may  also  enhance  convergence.  The 
method  also  avoids  the  tedious  calculation  of  the  upper  limit  of  the  integral  to  solve  for 
the  next  x.  in  (G.4  ). 

3.       NUMERICAL  RESULTS 

We  have  solved  for  the  quantizer  parameters  for  a  gaussian  random  variable  of 
zero  mean  and  unit  variance  for  several  values  of  N  up  to  128.  Also  the  mean  square 
error  D  and  the  output  entropy  (-^j^  Pj^  log2  (Pj^  ))  have  been  calculated.  The  results 
presented  in  Table  4  show  that  in  several  cases  Max's  results,  which  were  only 
available  up  to  N  =  36,  are  not  accurate  in  the  last  digit. 
Key  to  Table  4 

The  numbering  in  the  table  is  as  follows. 

1.  For  N  even,  each  table  begins  with  the  (N/2+  1)^  parameters.    In  this  case  the 
(N/2+  1)^  value  of  x  is  zero. 

2.  For  N  odd,  Each  table  begins  with  the  (N72  +  2)^  parameters.    In  this  case  the 
(N/2  +  2)|^  value  ofy  is  zero. 

Negative  parameters  can  be  calculated  from  the  symmetry  relation 

^i  =  X„.j^,  (G.ll) 

and 

1 

yi  =  y„.i-M-  (G-'2) 

A  FORTRAN  program  to  calculate  the  parameters  , distortion  and    entropy 
follows  Table  4.  The  only  input  to  the  program  is  N,  the  number  of  quantization  levels. 
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TABLE  4 
MAX'S  QUANTIZER  PARAMETERS  FOR  THE  NORMAL  DISTRIBUTION 


N  =   2  N  =       7 

j"'x""y  J       X         Y 

2    0.000000    0.797885  4   -0.280289    0.000000 

5    0.280288    0.560577 

ERROR    =   0.363380  6    0.874362    1.188147 
ENTROPY_= 1^000000 7    1.610758    2.033369 

__   _   _  ERROR    =   0.  044000 

J        X  Y       5^I^2£I_= i^iiMll- 

2  -0. 612003    0. 000000  N  =        8 

3  0.612003    1.224006    

.  —  —  _  —  •_.-___  —  —  —  _  —  —  _-  —  -  —  —  —  -.  J        X          Y 

ERROR    =   0.  190174  *--- 

ENTROPY  =   1.535789  5    0.000000    0.245094 

==========================  6    0.  500550    0. 756005 

N=        4  7    1.049957    1.343909 

•--■ 8    1.747927    2.151946 

J"        X  Y        —  —  —  —  —  —  —  —  —  —  —  —  —  —  —  «-._. 

ERROR    =   0. 034548 

3  0.000000    0.452780  ENTROPY  =   2.824865 

4  0.981599    1.510418  ========================== 

N  =        9 

ERROR    =   0.  117482  "- 

ENTR0PY_= 1:_511222 J        X          Y 

"~N~=        5  5   -0.221819    0. OOOOOo' 

6    0.221819    0.443639 

J        X          Y  7    0.681217    0.918796 
8    1.197594    1.476392 

3  -0.382284    0.000000  9    1.865528    2,254664 

4  0.382284    0.764567 --- 

5  1.244357    1.724147  ERROR    =   0.027853 
. EL^I52ZI_~   2.  982695 

ENTR0PY_= 2^202916 N  =       10 

.-        -        ___  0.000000    0. 199623~ 

.^. 7    0.  4C4740    0.  609857 

4  0.000000    0.317716  g    0.833841    1.057825 

5  0.658911    1.000106  9    1.324583    1.591340 

6  1.446850    1.893595  10    1.968218    2.345096 

ERROR    =""or057978"'  ERROR    =   0.022937*" 

ENTROPY  =   2.442789  ENTROPY  =   3.124584 
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TABLE  4 
MAX'S  QUANTIZER  PARAMETERS  FOR  THE  NORMAL  DISTRIBUTION  (CONT'D.) 


N  = 


11 


N  = 


14 


X 


X 


6  -0.183729    0.000000  8    0.000000    0.145706 

7  0.183729   "0.367458  9    0,293513    0.441321 

8  0.559913    0.752367  10    0.595882    0.750443 

9  0.965597    1.178826  11    0.918039    1.085635 

10  1.435733    1.692639  12    1.276582    1.467528 

11  2.059193    2.425746  13    1.703070    1.938612 
14    2.281837    2.625062 

ERROR    =   0.019220      

ENTROPY  =   3.253506  ERROR    =   0.012232 

:==========================  ENTROPY  =   3.582050 

N  =          12  =========================: 

--- - N  =       15 

J        X  Y         

J        X          Y 

7  0.000000    0.168438     

8  0.340142    0.511846  8   -0.136929    0.000000 

9  0.694313    0.876779  9    0.136928    0.273857 

10  1.081245    1.285711  10    0.414310    0.554764 

11  1.534371    1.783030  11    0.702949    0.851134 

12  2.140733    2.498435  12    1.013007    1.174879 
13    1.360468    1.546057 

ERROR    =   0.016340  14    1.776266    2.006474 
ENTROPY_= 3^371666 15    2.343670    2.680866 

N  =       13  ERROR    =   0.010737 

ENTROPY  =   3.676630 

J        X          Y  ========================== 

N  =       16 

7  -0.156887    0.000000     

8  0.156887    0.313773  J        X          Y 

9  0.476012    0.638251     

10  0.812600    0.986949  9    0.000000    0.128395 

11  1.184106    1.381263  10    0.258222    0.388048 

12  1.622890    1.864518  11    0.522404    0.656759 

13  2.214522    2.564525  12    0.799550    0.942340 
_ 13    1.099286    1.256231 

ERROR    =   0.014063  14    1.437139    1.618046 

ENTROPY  =   3.480744  15    1.843532    2.069017 

16    2.400803    2.732590 


ERROR 
ENTROPY  = 


0. 009501 
3. 765328 
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TABLE  4 
MAX'S  QUANTIZER  PARAMETERS  FOR  THE  NORMAL  DISTRIBUTION  (CONT'D.) 


<T> 


I  oaMnr>-vD»HiHionvD 
I  ooLncr>v£>ocr»'*r^rH 

I  ooooc7>rHCNinnr-iaD 

I  OrH'^UJiHC^VOOCOvD 
I  OCN"*VO<7>rH^COCNCO 

I  doOOOi-t«HiH<NCS  I 

I  I 

I  I 

I 
I 
I 
i 
I 


I  II 
I  II 
I  0000  II 

I  '^J'rHll 

I  com  II 

I  UDCVJ  II 
lOOII 

I  oo 


II 

O-^ll 


I 

I  Ln'*<NvOrH';j<vDoooMr) 

I  oooor-nrocrnOLr)'^ 

I  (Nojnor^cNCNLnnr-- 

X  I  CTiCTicriLnocNicoror^cri 

I  rHiHcoLDr^-OrovDOLn 
I    . 

I  OOOOOrHiHiHCSCN 
I     I 

I 

•d  I  O<HCNrT>^tnv0C^C0(T> 

I  rHi-Hi-ti-iiHcHiHiHi-irH 


I 


I  o<Tioococr>cr>'*r-MD 
I  OcnoocNLOcO'^^cnc^ 

>H  I  O^CTi^CsimOOCNOO 
I  OcN)'*r^Orov£)i-ir- 
I 

I  OOOOfHrHi-ICNCN 

i 

I 

I  t^r^00Lncvjvocnr~-<i) 
I  cncTinoo^'^'.oovo 

I  ^•^CT'O'^CNVDC^CO 

Xi  I  i-iiHvoor^oor^LOn 
I  cNjcNvocNCor-ooin 
I  t-^r-^cn<^C0r-^lD(J\^ 
I 

I  OOOOOrHiHrHCN) 

I     I 

I 

I 

I        i-ti-tiHi-lt-ti-lfHiH 


II  no 

II  <N 
><ll 
CMII 
I  DiOII 

I  oa  II 

I  OiHIl 
I  DiSlI 
I  WWII  II 

I        II 


x; 


I  I 

I  cvj«HooncMt^r}<r--'*«H  i 

I  VOCTiOOCT^^Dt-IC^rHvD   I  OOCO 

I  r^t^^invD^^a>r^cn  i  ooo 
I  ncNvDconoonvooor^  i  cnlo 
I  o rH cN'ct^ 00 ncN in r^o  i  ^c^ 
I  rHroir)r^cr>cNmoocNa»  i  or^ 
I lOO 

I  OOOOOrHrHiHCNCN  I  •  • 
I  I  O^ 

I  I 

I  or^OrHooinrHvDvor^  i  ii  ii 
I  or--<st<i-HOOLn'*cr>'*n  i 
I  ocMvOLnoocTK-icon  i  >h 
I  ooocrir^v£>rHOOr^n  i  (x, 
I  o o tH CO vo<H CO o^<^cr>  I  cc;o 
I  o<N'*<x)C0tH0TDOLn  I  Oct; 

I I  DiH 

I  OOOOOtHrHfHCNCN  I  LliS 


I 
I 
I 

I  rH  CM  n 'ct*  in  i£>r^  00  CT>0  i 

I  «-lf-irHrHr-liHfHi-1r-lCS    I 


I  WW 

I 

I 


I 

1     II 

1 

1     II 

1 

t     II 

1 1^ 

1  on 

1  vD 

1  '=:J^II 

1  ^ 

1  con 

1  CO 

1  con 

1  >i  1 

1  o 

1  <*ii 

1  o 
1    . 

1 00  n 
1    .11 

1  o 
1 

1 

n  II 

1     II 

II 

1  11 
1 

II   II  CO 

II  rH 
>^ll 

1       1 

CM  II 

X  I 

1  Ci   1 

Oil       1 

1  O  1 

Dill         1 

1  (X  1 

HII        1 

1  (X  I 

2  11        1 

1  W  1 
1      1 

Wll   II    1 
II        1 
IIS  1 

1      1 

II        1 

^  1 

1      1 

II        1 

1 

I 

cr>  incNCNOr^ror^r^  i 

^  -^OMDOfNOCM-H  I  rOfH 
r^  nrooOiHooocr^oo  i  cr>'^ 
'T  vD'*n<Ncr>uDOin  i  inc^ 

<-•  coincOrHnr^i-Hoo  i  ocn 
• I  ocr> 

O  000<-i>-trH(NCsJ  I     .    . 

I  Ocn 

O  r^-^cMiHcoinincvj  i  II  II 
o  incNCOcovOr-iujr^  i 
o  innocncTicri'CT'm  i  >h 
o  oin(nr^ocNmro  i  Oi 
O  rTvDO^Dinr^vOO  I  OiO 
O  cN^r^cncNincTiin  i  oc^ 

•     • I  CcJEh 

O  OCOOrHrHiHCM   I  (XZ 
I  WW 


O  rHCNn^LOi£>r-oo  i 

I— ii— li— li— ti— it— It— liH   I 
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TABLE  4 
MAX'S  QUANTIZER  PARAiMETERS  FOR  THE  NORMAL  DISTRIBUTION  (CONT'D.) 


CO 
CN 


I 
I 
I 
I 
I 
I 
I 
I 

I 

I  OOOOOO«HiHrHrHCVjr0 
I 


Ooooir)CM'*THt^oor~CNO 
ocoooi  rH  Lo  <^oocM  in  un 

OvO^nOvOLDOOLDCNiH 

o  iH  in  n  00  <N  iH  tH  rH  a>  lo  in 
o  00  vi)  in  "*  in  r^  i-i  00  cr»  o  rH 
o  rH  ro  in  r-- o^  <H  "^  ^  cr> -^  o 


I  >*'^':j<ooooror^'*inino«H 

I  '#^'^v£>v£)r0O00OvOCr>O 

I  oocoinrovDniHCNOcNrocN 

X  I  OOCOC^OOCNrHVDOCNO 

I  (3^(T>r-inLnLn'^cri'^^OrH 

I  OOCN'*v£>00O(Nin00CNr- 

I 

I  000000<-trHr-iiHCN<N 
i     I 

I 

»D  I  CMfT^inuDr-oocnOTHCNn 

I  i-HrHi-lTHrHrHrHi-lCNCSCSCS 


1          II 

1        II 

1       II 

1  vDiOIl 

1  ^Oll 

1  i>ooii 

1  ^r^ii 

>H    1 

1  OvOII 

IOCS  II 

1    .  .11 

lO^II 

1         II 

1         It 

1  II  II  II 

'^f 

1         II 

CN 

1       >h!I 

1       CM  II 

X  1 

1  OiOII 

1  OOiil    1 

1  DiH  II    1 

1  DiZII    1 

1  UUII    1 

II    1 

1            II    1 

1            II    1 

S  1 

1            II    1 

<->  1 

1            II    1 

II 

CSrHCOO^fHCN^^COOOOO 

r^  o  in  c^  CO  (T>  f-1  CO  vD  ^  cr>  (T> 
OtHcomoovDoovDcnoco 

r^  CN  CTKN  (N  CS  vO  iH  t^  .H  n  t^ 
00  VD  CO  CS  rH  iH  CS  ^i5  CM  "*  "*  ^ 
OCN^vOOOOC^'^C^O^O 

OOOOOiHrHiHrHCvjCNOO 


or^r-cMt^oocN 
ocot^tHcoooin 

Oin<T>rHCNOrn 
O^OiH[^cNcr> 
Or^incoiHi-tTH 
OiHcninr^cTirH 


notion  00 
csinocN'^J* 
cNr^ooinc^ 
-^^-^cNin 
•*cr>cO'*'* 
roinoocNr~- 


OOOOOOiHtHrHi-ICNCvq 


n  "^  in  vD  C^  00  <Ti  O  rH  cvj  ro '^ 

r-|iHrHrHiHr-|«HCNCNCNCN)C^ 


I 

I  CNJCM 

I  r--tH 

I  roiH 

I  '=:}'r- 

I  OCN 

I  Oro 

I  .    . 

I  O^ 

I 

I  II  II 

I 

I  >H 

I  Oh 

I  Ct^O 

I  OOi 

I  DiEH 

I  P^Z 

\  WW 

I 
I 
I 


CN 


I 
I 
I 
I 

>^  I 
I 
I 
I 
I 
I 
I 
I 

I 

X  I 
I 
I 
I 
I 
I 
i 
I 

•o  I 


or^-incNjcscN 
omtncTi'^in 
Crocnoo^in 

OOOCjMnrHO 

oci^cnocvjin 

OiHcovOOOO 


in-^rocTir-- 

CNr-iCNCOO 

o<Nnc\]>£) 
cr>cr>r--mLn 
CTir-ocvj"^ 
cNincr^cnat 


OOOOOrHfHr-irHCNCN 


cr>  00  li) -*  r- c^  CO  cn  cr>  v£>  00 
r- r~- in  <N  ^£>  C5>  n  vo  vo  o  "^ 

iH  iH  00  VD  Vi)  CTirH  "^  CN  n  ^ 

CTi  CTi  CO  cvi  ro  in  in  CT>  ro  in '^ 
(j>  (Ti  o^  O  iH  n  r^  n  "^  iH  ro 
O  O  cNj  in  r^  (j>  iH  •:;j' r^  1-t  vi) 

OCOOOOfHi-HtHCSJOq 

I 


«H  Cs!  0T:J<  in  <X>  r^  CO  (7>  O  iH 
r-i  i-H  iH  1-1  iH  i-i  i-i  r-i  rH  CN  CVJ 


I  II 

I  11 

i  II 

I  no  II 
I  in<T>  II 
I  >£>CV]  II 
I  intH  II 
I  O'^ll 

I  OrHll 

I    .  .11 

I  Ot-  !l 


II  II 


II 

II  CM 
II  CM 

I       >^ll 

I       CMII 

I  CiOII 

I  ODiil 

I  Ci^li 

I  0:2:  W 

I  WWII 

I  II 

I  II 

I  II 

I  11 

II 


II 


vD  00  CM  in  vDO^  rocs  crisis'* 
00  C3^roC0r^«H<7.CMrO00r^ 
vD  <H(N'^CMOOvi)C^nCM 

'^t*  ina>(ri<T>nr-»-i'^intH 
cr>  oor^r^oor-iinmin^Doo 
O  CM'^vDoOrHcovDOincn 

O  OOOOi-H'-''-i'-fCMCM 


O  CMinCTiOC^vOOOOCMO 

O  ^  1-1  in  00"=:?*  in  in  00  vom 
O  CTiCNcncorHOroiHOro 
O  C^CN(7>'*rHLn':^nOro 
O  OOCOC^COOroCTiCri^Dr- 
O  iHcoinr^ocM^r-i-HvD 

O  OCOOrHi-lrHrHCNCM 


CM  m^in^Dr^cocTiOiHCM 

rH  «HrHrHT-lt-1iHrHCMCNCN 


OCM 

iHcr> 
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00 

OCM 


II  11 

QiO 

00:^ 

WW 


129 


TABLE  4 
MAX'S  QUANTIZER  PARAMETERS  FOR  THE  NORMAL  DISTRIBUTION  (CONT'D.) 


N  = 


25 


J 

X 

Y 

13 

-0. 083805 

0. 000000 

14 

0. 083805 

0. 167610 

lb 

0.252208 

0. 336806 

lb 

0. 423045 

0. 509283 

17 

0. 598128 

0.  686972 

Id 

0. 779592 

0. 872212 

ly 

0. 970115 

1. 068019 

2U 

1. 173279 

1.  278540 

21 

1.394213 

1. 509886 

22 

1. 640881 

1. 771876 

2J 

1. 927050 

2. 082224 

24 

2. 280667 

2. 479110 

2b 

2. 778634 

3. 078159 

ERROR 

0. 004041 

ENTROPY  = 

4. 384064 

====: 

===========. 

N 

26 

J 

X 

Y 

14 

0. 000000 

0. 080593 

15 

0. 161536 

0.  242480 

16 

0. 324498 

0. 406516 

17 

0. 490402 

0. 574288 

18 

0. 660961 

0. 747635 

19 

0. 838229 

0. 928823 

20 

1.024813 

1. 120803 

21 

1.  224230 

1. 327657 

22 

1. 441544 

1. 555432 

23 

1. 684648 

1. 813865 

24 

1. 967207 

2. 120549 

25 

2. 316997 

2. 513445 

2b 

2. 810502 

3. 107559 

ERROR    = 

0. 003746 



ENTROPY  = 

4. 438843 

N 

27 

J 

X 

Y 

14 

-0.077781 

0. 000000 

15 

0.077780 

0. 155561 

16 

0.233975 

0.312389 

17 

0.392106 

0. 471823 

18 

0. 553594 

0. 635364 

19 

0. 720073 

0. 804782 

20 

0. 893532 

0.982281 

21 

1.076518 

1. 170756 

22 

1.272495 

1.374235 

23 

1. 486469 

1. 598704 

24 

1. 726267 

1. 853829 

25 

2. 005461 

2. 157093 

26 

2. 351670 

2. 546247 

27 

2. 840977 

3. 135707 

ERROR 

0.003483 

ENTROPY  = 

4. 491610 

N 

28 

J 

X 

Y 

15 

0. 000000 

0. 075012 

16 

0. 150307 

0. 225602 

17 

0, 301760 

0. 377919 

18 

0. 455569 

0.533219 

19 

0. 613076 

0. 692934 

20 

0. 775854 

0. 858775 

21 

0.945836 

1.032897 

22 

1. 125522 

1. 218147 

23 

1.  318326 

1. 418505 

24 

1. 529205 

1. 639905 

25 

1. 765925 

1. 891945 

26 

2.041975 

2. 192005 

27 

2. 384821 

2. 577637 

28 

2. 870169 

3. 162701 

ERROR 

0. 003246 

ENTROPY  = 

4.  542507 
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TABLE  4 
MAX'S  QUANTIZER  PARAMETERS  FOR  THE  NORMAL  DISTRIBUTION  (CONT'D.) 


N  =      29  N  =       31 

J       X         y""    '  J  X  Y 

15'"In"n79R^^ n'nnnnnn"  ^^  -0.068008  0.000000 

ifi    n  n-T^m    n-?9§?§§  17  0.068008  0.136016 

17  0'9?R9ff    n'^af^i?  18  0.204446  0.272876 

18  nil^l^l        n'lUit^  19  0.342170  0.411464 

19  oJfst^R    n-tofffZ  20  0.482101  0.552739 

20  n  ifiq???    n'^mio  21  0.625267  0.697794 

21  O^ifai^P    n-QnQQ9?  22  0.772858  0.847921 

22  0  9954^?    ?-n2nq^Q  23  0.926315  1.004710 

23  1  ?79n7A    f§i??nQ  24  1.087456  1.170202 

24  1  361940    Mini??  25  1.258670  1.347138 

25  1  siqQA?    i-iiooJ}  26  1.443261  1.539385 

26  1  ao?7Rft    I'Qot^ii  27  1.646065  1.752745 

27  2  076890    9' 9 9?!?!  28  1.874694  1.996643 

28  9  diA^7?  oin^ti^  29  2.142413  2.288184 
9Q  9fiqfti77  ?  iqZZ??  30  2.476285  2.664385 
t: rif-I.ZZ _li??5  31  2.950981  3.237577 

-L:. 30 -  ---7^-       32 

18  0.282019    0.353110  18  0.131971  0.198052 

19  0.425412    0.497714  19  0.264715  0.331378 

20  0.571795    0.645876  20  0.399039  0.466699 

21  0.722402  0.798927  21  0.535816  0.604934 
??  ?-^Z§Z§i  ?-?§?^90  22  0.676035  0.747136 
9?    f§fi?§?    1-126640  23  0.820850  0.894565 

24  1.216393  1.306147  24  0.971674  1.048783 
?i    }'i^lP.9         }•  500912  25  1:  130294  1.211804 

25  1.608846    1.  /16779  26  1.299072  1.386340 

U         ^'ftS^SJ    i-?f???^  27  i: 481284  1.576228 

o§    2.110332    2.257440  28  1.681731  1.787233 

in        i'ttZSiZ    ?-§??fJ4  29  1.907981  2.028728 

30    2.925088    3.213562  30  2.173234  2.317739 

— ^DDno ;;';;;;;o;;;"-"  3i  2.504429  2.691120 

ig?g§PY  =    W3W9I  lL.J.-J.lltlL.J.ilt2llL 

ERROR    =  0. 002505 

ENTROPY  =  4. 729784 
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TABLE  4  .  •■^" 

MAX'S  QUANTIZER  PARAMETERS  FOR  THE  NORMAL  DISTRIBUTION  (CONT'D:) 


N  = 


64 


J" 

X 

^  «■  ■»  ■ 

Y 

33" 

0. 

"000000' 

""0! 

"033409" 

34 

0. 

066844 

0. 

100278 

35 

0. 

133787 

0. 

167297 

36 

0. 

200932 

0. 

234567 

37 

0. 

268380 

0. 

302193 

38 

0. 

336238 

0. 

370283 

39 

0. 

404616 

0. 

438950 

40 

0. 

473632 

0, 

508314 

41 

0. 

543408 

0. 

578503 

42 

0. 

614079 

0. 

649655 

43 

0. 

685789 

0. 

721922 

44 

0. 

758595 

0„ 

795468 

45 

0. 

832972 

0, 

870476 

46 

0. 

908816 

0. 

947155 

47 

0. 

986446 

1. 

025736 

48 

1. 

066112 

1. 

106488 

49 

1. 

148104 

1. 

189720 

50 

1. 

232757 

1. 

275794 

51 

1. 

320468 

1. 

365141 

52 

1. 

411709 

1. 

458276 

53 

1. 

507054 

1. 

555831 

54 

1. 

607210 

1. 

658589 

55 

1. 

713065 

1. 

767542 

56 

1. 

825759 

1. 

883977 

57 

1. 

946794 

2. 

009611 

58 

2. 

078211 

2. 

146810 

59 

2. 

222896 

2. 

298981 

60 

2. 

385143 

2. 

471305 

61 

2. 

571789 

2. 

672274 

62 

2. 

794840 

2. 

917407 

63 

3. 

078922 

3. 

240437 

64 

3. 

492269 

3. 

744101 

ERROR 

"or000644~~ 

ENTROPY  = 

5. 710078 

132 


TABLE  4 
MAX'S  QUANTIZER  PARAMETERS  FOR  THE  NORMAL  DISTRIBUTION  (CONT'D.) 


N  = 


128 


J" 

X 

- 

11' 

-0. OOOOOl' 

0. 016823~~ 

5(5 

0. 033559 

0. 050490 

67 

0.067331 

0. 084172 

68 

0. 101029 

0. 117836 

69 

0. 134765 

0. 151644 

19 

0. 168552 

0. 185460 

71 

0.202404 

0.  219347 

72 

0.236333 

0.  253318 

73 

0.270353 

0. 237387 

74 

0.304477 

0. 321567 

75 

0. 338720 

0. 355874 

76 

0. 373C97 

0. 390320 

77 

0. 407620 

0. 424921 

78 

0. 442307 

0. 4596°3 

79 

0. 477172 

0. 494651 

80 

0. 512231 

0.  529812 

81 

0. 547502 

0. 505102 

82 

0. 583001 

0. 600810 

83 

0. 618747 

0. 636684 

84 

0. 654758 

0. 672833 

85 

0.  691055 

0. 709277 

86 

0. 727658 

0. 746039 

87 

0. 764589 

0. 783139 

88 

0. 801871 

0. 820602 

89 

0. 839528 

0. 858454 

90 

0.877587 

0. 896719 

91 

0. 916074 

0. 935428 

92 

0.955019 

0. 974609 

93 

0. 994453 

1. 014296 

94 

1. 034409 

1. 054523 

95 

1. 074924 

1. 095326 

96 

1. 116037 

1.  1367J.7 

97 

1. 157788 

1. 173823 

98 

1. 200223 

1. 221617 

99 

1.  243391 

1. 265165 

100 

1. 287346 

1. 309527 

lOi 

1. 332146 

1. 354766 

102 

1.377857 

1. 400948 

103 

1. 424543 

1. 448143 

104 

1. 472299 

1. 496450 

105 

1. 521196 

1.  545943 

106 

1. 571339 

1. 596733 

107 

1. 622833 

1. 648933 

103 

1. 675805 

1. 702677 

105 
110 

1. 730394 
1. 736759 

1. 753111 
1. 815407 

111 

1.845085 

1. 874762 

112 

1. 905583 

1. 936404 

113 

1.966503 

2. 000601 

114 

2.034136 

2. 067671 

115 

2. 102332 

2. 137993 

116 

2. 175010 

2. 212077 

117 

2. 25118J 

2. 290339 

113 

2. 3319S6 

2.  3736.;4 

119 

2. 413222 

2.  4f>2'51  1 

120 

2. 510926 

2. 5590 iO 

121 

2. 611463 

2. 663885 

122 

2. 721700 

2.  77951 i 

123 

2. 844280 

2. 909047 

124 

2.983146 

3. 057246 

125 

3. 144589 

3. 231933 

126 

3. 339681 

3. 447430 

127 
128 

3. 591183 
3. 962315 

3. 734936 
4. 189694 

ERROR 
ENTROPY  = 


0. 000163 
6. 699533 
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4.        PROGRAM  LISTING  TO  CALCULATE  THE  LLOYD-MAX  QUANTIZER 
PARAMETERS 

C  THIS  PROGRAM  CALCULATES  LLOYD-MAX  QUANTIZER  PARAMETERS  BY  THE  METHOD 

C  OF  SUCCESSIVE  SUBSTITUTION  FOR  THE  NORMAL  DISTRIBUTION  OF  ZERO  MEAN 

C  AND  UNIT  VARIANCE 

C  The  INPUT  TO  THE  PROGRAM  IS 

C    (1)     THE  NUl-IBER  OF  QUANTIZATION  LEVELS   N 

C    (2)     THE  MAXIMUM  NUMBER  OF  ITERATIONS    M 

C    (2)     THE  ACCURACY  AP 

REAL*8  X(199),T(199),XX(199),TT(199),C  , DELTA, AP( 199 ) ,AP 
1 , ERROR , ENTROP 
INTEGER  K,N,I,P,N1,N2,N3,M 
C=DSQRT(00.50D0/DATAN(1.0D0)) 
C     DO  99  N=110,110 

C   INPUT  THE  NUMBER  OF  QUANTIZATION  LEVELS 
N=100 

WRITE(9,65)N 
WRITE(9,66) 
WRITE(9,67) 
WRITE (9, 66) 

C     INITIALIZATION  OF  THE  QUANTIZER  PARAMETERS 
C 

DELTA=0.0150D0*DFLOAT(N) 

XX(1)=-10.50000D0 

TT(1)=-5.50000D0 

X(1)=XX(1' 

T(1)=TT(1 

DO  50   L=2,N 

TT(L)=TT(L-1)-DELTA 
XX(L)=(TT(I)+TT(I-1))/2.0D0 
X(L)=XX(L) 
T(L)=TT(L) 
50    CONTINUE 

C    BEGINING  OF  THE  ITERATIONS 
C     M  =  MAXIMUM  NUMBER  OF  ITERATIONS 
M  =  1050 
K=0 
5    K=K+1 

IF  (K  .GT.  M)  GO  TO  10 
IF  (K  .GT.  1)X(N)=XX(N 


IF  (K  .GT.  1)T(N)=TT(N 


TT  ( 1  )=-C'^DEXP  ( -XX  ( 2 )  '^XX  ( 2 )  /2 ,  ODD )  /  (DERFC  ( -10.  ODO )  - 
1DERFC(XX(2)/DSQRT(2.0D0))) 
T(l)=TT(i) 

IF  (  N  .EQ.  2  )  GO  TO  17 
DO  15   P=2,N-1 

XX(P)=(T(P)+T(P-1))/2.0D0 

X(P)=XX(P) 

TT(P)=DEXP(-X(P)*X(P)/2.0D0)-DEXP(-X(P+1)*X(P+1)/2.0D0) 

TT (P)=TT(P)*C/ (DERFC (X(P)/DSQRT (2. ODO)) -DERFC (X(P+1)/DSQRT (2.0 


1      ODO))) 

T(P)=TT(P) 
15  CONTINUE 

17  CONTINUE 

XX(N)=(TT(N)+T(N-1))/2.0D0 
TT(N)=DEXP(-XX(N)*XX(N)/2.0D0)*C/DERFC(XX(N)/DSQRT(2.0D0)) 
X(N)=XX(N) 
T(N)=TT(N) 

N2=IDINT ( DFLOAT ( (N+2 )/2 
N1=IDINT(DFL0AT( (N+l)/2' 

C    CHECKING  THE  PRECISION  OF  THE  SOLUTION 
C    AP  =  REQUIRED  ACCURACY 
AP=0.10D-6 
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IF((M0D(N,2)  .EQ.  0) .AND . (DABS (X(N2) ) .GT.  AP))GO  TO  5 
IF((M0D(N+1,2)  .EQ.  0).AND.  (DABS(T(N1) ) .GT.  AP))GO  TO  5 
CONTINUE 
10      CONTINUE 

C     OUTPUT  RESULTS 

IF  (M0D(N,2)  .EQ.  0)N3=N2 
IF  (M0D(N+1,2)  .EQ.  0)N3=N1 
WRITE(6,60)  K 
DO  120  J=1,N3 

IF  (J  .EQ.  1) 
1     WRITE(9,71)J,  T(J) 

IF  (J  .GT.  1) 
1     WRITE(9,61)J,  X(J),T(J) 
120    CONTINUE 

X(N+1)=10.0D0 
X(1)=-10.0D0 
ERROR=0.0D0 
ENTROP=0.0D0 
DO  222  1=1, N 

AP(I)=DERFC(X(I)/DSQRT(2.0D0))-DERFC(X(I+1)/DSQRT(2.0D0)) 
AP(I)=AP(I)/2.0D0 
ERROR=ERROR+AP  ( I )  '^T  ( I )  *'^2 

ENTROP=ENTROP-AP(I)'^DLOG(AP(I))/DLOG(2.0D0) 
222    CONTINUE 

ERROR=1.0DQ-ERR0R 
WRITE(9,66) 
WRITE(9,62)  ERROR 
WRITE(9,66) 
WRITE (9, 63)  ENTROP 
WRITE(9,66) 
WRITE(6,72)  K 
WRITE (9, 90) 
WRITE(9,66) 
C99    CONTINUE 

65  FORMAT (3X, '   N  =  ',17) 

66  F0RI1AT(3X,  ' '  ) 

67  FORMAT (3X,'  J       X         Y      ') 

60  F0RMAT(1X,I7,8(1X,F6.4)) 

90    FORMAT ( 2X , ' =========================================== ' ) 

61  F0RMAT(1X,I4,  2(2X,F9.6)) 

71  FORMAT (IX, 14, IIX,  2(2X,F9.6)) 

62  FORMAT (7X, 'ERROR   =',  2(1X,F9.6)) 

72  F0RMAT(3X,'#  ITERATIONS  =  ',17  ) 

63  FORMAT (7X, 'ENTROPY  =' ,  2(1X,F9.6)) 

STOP 
END 


135 


LIST  OF  REFERENCES 


[1.]  Lawson,  J.  "An  Elementary  Theory  of  Surveillance,"  Naval  Electronics  System 
Command,  Technical  Report,  Aug.  1978. 

[2.]  Betsekas,  D.  P.,  Tsitsiklis,J.  N.  and  Athans,  M.,  "Convergence  Theories  of 
Distributed  Iterative  Process:  A  Survey,"  Laboratory  of  Information  and 
Decision  Systems,  iVIIT,  Report  No.   LIDS-P-1342,  Dec.  1983. 

[3.]  Yemini,  Y.  and  Lazer,  A.,  "  Towards  Distributed  Sensor  Networks,  an  Extended 
Abstract,"  Proceedings  of  Information  Sciences  and  Systems  ,  1982,  pp.  198-302. 

[4.J  Lauer,  G.,  and  Sandell,  N.  R.  "Distributed  Detection  of  Known  Signal  in  Noise," 
Report  160,  ALPHATECH,  Burlington,  Mass.,  Mar.  1982. 

[5.]  Tenney,  R.  R.,  and  Sandell,  N.  R.  "Detection  with  Distributed  Sensors,"  IEEE 
Trans,  on  Aerospace  and  Electronic  Systems,  vol.  ASE-17,  4,  Jul.  1981,  pp. 
501-510. 

[6.]  Van  Trees,  H.  L.,  Detection,  Estimation,  and  Modulation  Theory-Part  I,  Wiley  and 
Sons,  New  York,  1969. 

[7.]  Sarma,  V.  V.  S.,  and  Rao,  K.  A.  Gopala,  "Decentralized  Detection  and 
Estimation  in  Distributed  Sensors  Systems,"  In  Proceedings  of  the  IEEE 
Cybernetics  and  Society  Conference,  vol-1,  1983,  pp.  438-441. 

[8.]  Chair,  Z.,  and  Varshney,  P.  K.,  "Optimal  data  fusion  in  multiple  sensor  detection 
systems,"  IEEE  Transaction  on  Aerospace  and  Electronic  Systems,  vol.  AES-22,  1, 
Jan.1986,  pp.  98-101. 

[9.]  Reibman,  A.  R.  and  Nolte,  L.  W.  "Optimum  Detection  and  Performance  of 
Distributed  Sensor  System,"  IEEE  Transaction  on  Aerospace  and  Electronic 
Systems,  vol.  AES-23,  1,  Jan.  1987,  pp.  24-30. 

[10.]  Ekchian.  L.  K.,  Optimal  Design  of  Distributed  Detection  Networks,  Ph.D. 
Dissertation,  MIT,  Sep.  1982. 

[11.]  Ekchian,  L.  K.  and  Tenney,  R.  R.,  "Detection  Networks,"  Proceedings  of  the  21st 
Conference  on  Decision  and  Control,  1982,  pp.  686-691. 


136 


[12.]  Papastavrou,  J.  D.  and  Athans,  M.,  "A  Distributed  Hypothesis-Testing  Team 
Decision  Problem  with  Communications  cost,"  Proceedings  of  the  25st  Conference 
on  Decision  and  Control,  Athens,  Greece,  Dec.  1986,  pp.  219-225. 

[13.]  C.  H.  Papadimitriou  and  K.  Steigliz,  Combinatorial  Optimization:  Algorithms  and 
Complexity,  Englewood  CliiTs,  NJ:  Prentice-Hall,  1982. 

[14.]  Tsitsiklis,  J.  N.  and  Athans,  M.,  "On  the  Complexity  of  Decentralized  Decision 
Making  and  Detection  Problems,"  IEEE  Trans,  on  Automatic  Control,  vol. 
AC-30,  No.5,  iMay  1985,  pp.  440-446. 

[15.]  Kovatana,  T.,  "Theoretical  analysis  of  instruction  alarm  using  two 
complementary  sensors,"  Final  Technical  Report,  Stanford  Research  Institute, 
iVIenlo  Park,  Calif,  Feb.  1973. 

[16.]  Fefjar,  A.,  "Combining  techniques  to  improve  security  in  automated  entry 
control,"  In  Proceedings  of  the  1973  Carnahan  Conference  on  Crime 
Countermeasures,  May.  1978. 

[17.]  Stearns,  S.  D.,  "Optimal  detection  using  multiple  sensors,"  Proceedings  of 
Carnahan  conference  on  security  technology,  Uxington,  KY,  USA,  1983. 

[18.]  Max,  J.,  "Quantization  for  Minimum  Distortion,"  IRE  Trans,  on  Inform.  Theory, 
vol.  IT-6,1960,  pp.  7-13. 

[19.]  Pearson  ,C.  E.,  Numerical  Methods  in  Engineering  and  Science,  Van  Nostrand 
Reinhold  Company,  New  York,  1986,  pp.  11-12  and  49-50. 

[20.]  Wozencraft,  J.  M.  and  Jacobs,  I.  M.,  Principles  of  Communication  Engineering, 
Wiley,  New  York,  1967. 

[21.]  E.  K.  Al-Hussaini  and  A.  .VI.  Al-Bassiouni,  "Performance  of  an  ideal  switched 
diversity  receiver  for  NCFSK  signals  with  Nakagami  fading,"  Trans.  lECE  Japan, 
vol.  E65,  Dec.  1982,  pp.  750-751. 

[22.]  E.  K.  Al-Hussaini  and  A.  M.  Al-Bassiouni,  "Performance  of  MRC  diversity 
Systems  for  the  detection  of  signals  with  Nakagami  fading,"  IEEE  Trans,  on 
Commun.   vol.  COM-33,  No.  12,  Dec.  1985,  pp.  1315-1319. 

[23.]  Lloyd,  S.  P.,  "Least  Squares  Quantization  in  PCM,"  Unpublished  Bell 
Laboratories  manuscript,  July,  31,  1957;  also  in  IEEE  Trans,  on  Inform.  Theory, 
vol.  IT-28,  pp.  129-137,  1982. 

[24.]  Moose, P  M  and  Al-Bassiouni,A.M  "Solution  of  the  Lloyd-Max  Quantizer 
Parameters  by  the  Method  of  Successive  Substitution,"  Naval  Postgraduate 
School  Technical  Report  No.  NTS  62-88-004. 


137 


[25.]    Anthony  D.  Walen,  Detection  of  Signal  in  Noise,  Academic  Press,  1971,  p.  366. 

[26.]  Sandell,  N  R.,  Jr.,  Variaya,  P.  C.  Athans,  M.  and  Safonov,  M.  G.,  "Survey  of 
Decentralized  Control  Methods  for  Large  Scale  Systems,"  IEEE  Trans,  on 
Automatic  Control,  vol  ,AC-23,  1978,  pp.  108-128. 

[27.]  Yu-Chi  Ho  and  K  Al-Ching  Chu,  "Team  Decision  Theory  and  Information 
Structures  in  Optimal  Control  Problems-  Part  I,"  IEEE  Trans,  on  Automatic 
Control,  vol.  AC- 17,  No.  1,  Feb.  1972. 

[28.]  Yu-Chi  Ho  and  K  Al-Ching  Chu,  "Team  Decision  Theory  and  Information 
Structures  in  Optimal  Control  Problems-  Part  I,"  IEEE  Trans,  on  Automatic 
Control,  vol.  AC- 17,  No.  1,  Feb.  1972. 

[29.]  Yu-Chi  Ho  and  Marcia  P.  Kastner,  "Teams,  Signaling  and  Information  Theory," 
IEEE  Trans,  on  Automatic  Control,  vol.  AC-23,  No.  2,  April  1978,  pp.  305-312. 

[30.]  R.  Srinivasan,  "Distributed  Rader  Detection  Theory,"  lEE  Proceedings,  vol.  133, 
F,  No.  1,  Feb  1986,  pp.  55-60. 

[31.]  David  A.  Castanon  and  Demosthenis  Teneketzis,  "Nonlinear  Data  Fusion,"  Proc. 
1982  MIT;  ON R  Workshop  on  Command  and  Control,  iMonterey,  CA,  August 
1982,  pp.  224-230. 

[32.]  G.  S.  Lauer,  D.  Teneketzis.  D.  A.  Castanon  and  N.  R.  Sandell,  Jr.  "Recent 
Advances  in  Distributed  Detection,"  Proc.  J 982  MITlONR  Workshop  on 
Command  and  Control,  Monterey,  CA,  August  1982,  pp.  175-179. 

[33.]  D.  Teneketzis  ,  "The  Decentralized  Quickest  Detection  Problem,"  In  proceedings 
of  the  21st  IEEE  Conference  of  Decision  and  Control,  Fort  Lauderdale,  FL,  1982, 
pp.  673-679. 

[34.]  Sophocles.J.  Orfanidis,  Optimum  Signal  Processing,  An  Introduction,  MacMillan 
Publishing  Company,  1985,  p.  8. 

[35.]  Tom  M.  Apostol,  Mathematical  Analysis,  Addison-Wesley  Publishing  Company, 
1974,  p.  14. 

[36.]  David  G.  Messerschmit,  "Quantization  for  Maximum  Output  Entropy",  IEEE 
Trans,  on  Infor.  Theory,  Sep.  1971,  pp.  612. 

[37.]    Gallagar,  R.,  Information  Theory  and  Reliable  Communication,  Wiely,  1969. 

[38.]  Y.  Bar-Shalom,  "Tracking  .VIethods  in  a  Multitarget  Environment,"  IEEE  Tran. 
Automat,  Contr.,  vol.  AC-23,  Aug.  1978,  pp.  618-626. 


138 


[39.]    D.    B.    Reid,    "An   Algorithm   for   Tracking    Multiple    Targets,"    IEEE   Trans. 
Automat,  Contr.,  vol.  AC-24,  Dec.  1979,  pp.  843-854. 

[40.]    Helstrom,  C.  W.,  Probability  and  Stochastic  Processes  for  Engineers,  Macmillan 
Publishing  Company,  1984,  pp.  179-181. 

[41.]    Kassam,  S.  A.,  "Optimal  Quantization  for  Signal  Detection",  IEEE,  Trans,  on 
Communication,  vol.  COM-25,  No.  5,iMay  1977,pp.  479-484. 

[42.]    Wood,  C.  R.,  "On  Optimum  Quantization",  IEEE  Trans,  on  Information  Theory, 
vol.  IT- 15,  No.2,  March  1969. 

[43.]    Velichkin,  A.  I.,  "Optimum  Characteristics  of  Quantizers,"   Telecommunication 
Radio  Eng.  (USSR),  vol.  18,  Feb.  1963,  pp.  1-7. 

[44.]   Garmash,V.    A.,    "The    Quantization    of   Signals    with    Non-uniform    Steps," 
Telecommunications,  No.  10,  pp.  10-12,  Oct.,  1957. 


139 


INITIAL  DISTRIBUTION  LIST 

No.   Copies 

1.  Defense  Technical  Information  Center  2 
Cameron  Station 

Alexandria,  VA  22304-6145 

2.  ■    Library,  Code  0142  2 

Naval  Postgraduate  School 
Monterey,  CA   93943-5002 

3.  Department  Chairman,  Code  62  1 
Naval  Postgraduate  School 

Monterey,  California  93943-5000 

4.  Prof  Paul  H.  Moose,  Code  62-Me  5 
Naval  Postgraduate  School 

Monterey,  California  93943-5000 

5.  Prof  John  M.  Wozencraft  ,  Code  62-me  1 
^    6560  Brookdale  Dr 

Carmel,  California  93943 

6.  Prof  Donald  P.  Gaver  Code  55-Gv  1 
Naval  Postgraduate  School 

Monterey,  California  93943-5000 

7.  Prof  Harriett  B.  Rigas  1 
Dept.  of  Electrical  Engineering  and  System  Science 

Michigan  State  University 
East  Lansing,  Michigan  48824 

8.  Prof  Arthur  L.  Schoenstadt  Code  53-Zh  1 
Naval  Postgraduate  School 

Monterey,  California  93943-5000 

9.  Prof  Charles  W.  Therrien  Code  62-Ti  1 
Naval  Postgraduate  School 

xMonterey,  California  93943-5000 

10.  Prof  Daniel  C.  Bukofzer  Code  62-Bh  1 
Naval  Postgraduate  School 

Monterey,  Cahfornia  93943-5000 

11.  Prof  Roberto  Cristi  Code  62-Cx  1 
Naval  Postgraduate  School 

Monterey,  California  93943-5000 


140 


12.      Prof.  Louis  L.  Scharf 

Dept.  of  Electrical  and  Computer  Engineering 
University  of  Colorado 
Boulder,  CO  80309 

12.      Prof.  Richard  A.  .McGonigal  Code  0305 
Naval  Postgraduate  School 
Monterey,  California  93943-5000 

14.  Prof.  Donald  Trahan  Code  53-Tn 
Naval  Postgraduate  School 
Monterey,  California  93943-5000 

15.  Prof  Richard  Franke  Code  53-Fe 
Naval  Postgraduate  School 
Monterey.  California  93943-5000 

16.  Prof  Carroll  Wijde  Code  53-\Vm 
Naval  Postgraduate  School 
.Monterey,  California  93943-5000 

17.  Mr.  Larr}'  Frazier  code  0141 
Naval  Postgraduate  School 
Monterey,  California  93943-5000 

18.  Prof  Emad  K.  Al-Hussaini 

EE  Dept.,  Faculty  of  Engineering 
P.O.  15551,  UAE  University.  Al-.\in 
United  .A.rab  Emirates 

19.  Prof  Abdel-Wahaab  Fayez 

Faculty  of  Engineering,  Cairo  University 
Giza,  Egypt 

20.  Prof  P.  K.  Varshney 

Electrical  and  Computer  Engineering  Department 
111  Link  Hall.  Syracuse  University 
Syracuse,  NY  13210 

21.  Dr.  Dave  Harris  L-205 

Lawrence  Livermore  NTL  Laborator}' 
P.O.  Box  808 
Livermore,  CA  94550 

22.  Chief,  Research  and  Advancement  Authority 
Egyptian  Armed  Forces  Headquarters 
Nasr  City,  Cairo,  Egypt 

23.  Chief,  Armament  Authority 

Armament  Authority  of  Egyptian  Armed  Forces 
Koprey  El-qubaa,  Cairo,  Egypt 


141 


24.  Chief,  Training  Authority 
Egyptian  Armed  Forces  Headquarters 
Nasr  City,  Cairo,  Egypt 

25.  Director,  Signal  Corps 

Egyptian  Armed  Forces  Headquarters 
Nasr  City,  Cairo,  Egypt 

26.  Library',  Military'  Technical  College 
Koprey  El-qubaa 

Cairo,  Eg\'pt 

27.  Librar\',  Faculty  of  Engineering 
Cairo  University 

Giza,  Egypt 

28.  Lieutenant  Sarfraz  Hussain 
1577/18  Samnabad 
Karachi,  Pakistan 

29.  Mr.  Seah  Moon  Ming 

Blk.  IF,  Gillman  Heights  #10-51 
Republic  of  Singapore 

30.  Brigadier  General  Magdy  G.  El-Sheimy 

317,  King  Faesal  St.,  Madkor  Station,  Pyramid 
Giza,  Egypt 

31.  Colonel  Salah  M.  El-Magraby 
Building  38,  Apt.  21.  Tawfeek  City. 
Nasr  City,  Cairo,  Egypt 

32.  Colonel  Abdel-Moniem  E.  Doma 
45,  Ibraheim  Abdel-Razik  St. 
Ein  Shams,  Cairo,  Egypt 

33.  Colonel  Mohamed  M.  .Abdel-Moniem 
45,  Ibraheim  Abdel-Razik  St. 

Ein  Shams,  Cairo,  Egypt 

34.  Colonel  Ismaeel  A.  Al-Getani 

52,  Bochar}'  St.,  Enterance  2,  Apt.  5 
The  New,  7(th)  Area 
Nasr  City,  Cairo,  Egypt 

35.  Colonel  Refat   M.  El-Hefny 
23,  High  Industrial  Institute  St. 
Ein  Shams  (east),  Cairo,  Egypt 

36.  Colonel  Abdel  Aziz  M.  Al-Bassiouni 
1  Hamazeen  St. 

Kafrel  Zaiat,  Egypt 


142 


3^:i  '  393 


Thesis 

A.3347   Al-Bassiounx 

^  1       Optimum  signal  process- 
ing in  distributed  sens 
systems. 


or 


Thesis 

A3347   Al-Bassiouni 

c.l  Optimum  signal  process- 
ing in  distributed  sensor 
systems. 


ff 


jopi 


thesA3347 

Optimum  signal  processing  in  distributed 


3  2768  000  80285  4 

DUDLEY  KNOX  LIBRARY 


lllHdli 


m 


11 


